Integrating Complementary Feature Sets for Human-AI Decision-Making

In many AI-assisted decision making settings, a machine learning (ML) model and a user may have access to different sources of information (features), both of which are required to make accurate decisions. To combine these complementary features into a single, collaborative decision, either the user or the ML model must do the work of integrating the other decision maker’s features into their decision. Most current human-ML decision-making settings rely on the user to do this work by having the ML model generate a prediction that the user considers while making their decision. Some recent work has proposed an approach for communicating user features to an ML model which simplifies the user’s decision-making task, but requires the user to communicate their features to the model. These approaches have very different user experiences that may have different effects on the resulting decision. In this work, we conduct a user study to explore tradeoffs between these approaches in terms of decision accuracy, response time and user experience when users communicate complementary features to an AI through a text-entry interface. We also study how users navigate tradeoffs in this setting when allowed to choose between the approaches for each instance and compare this to a simple adaptive strategy that automatically switches between the two approaches based on estimated instance difficulty.