Tough Ratings, Tougher Sell: How Different Types of Adjustment Affect Managers’ Asymmetric Algorithm Use in Performance Evaluation Judgments

ABSTRACT Despite the potential of algorithms to improve judgment quality, recent research suggests that individuals may be averse to algorithmic use. We experimentally examine whether and how managers’ use of an algorithm-advised performance rating is influenced by rating valence and the decision rights managers have to adjust the algorithm. We find that managers are less willing to use an algorithm to evaluate subordinate performance when it advises a low, rather than high, rating. We further show that when the algorithm-advised rating is low, allowing managers to adjust how the algorithm computes the rating, compared with adjusting the rating itself or not allowing any adjustment, increases algorithmic use. Further analyses show this effect to be consistent with managers’ increased understanding of an algorithm when involved in its computation. Our findings inform organizations’ implementation of performance evaluation algorithms by showing how rating valence and decision rights jointly influence managers’ use of the algorithms.