Balancing automation and control: user perceptions of tool invocations in conversational agents

Abstract Advances in large language models (LLMs) have enabled conversational agents (CAs) to invoke external functions dynamically, yet little is known about how different tool-calling (TC) strategies affect user experience. This study compares two approaches – automatic execution based on conversational context versus user-confirmed execution – using a German-language CA in a business-travel scenario. In a between-subjects experiment, 451 employed adult participants (aged 18–65) were randomly assigned to one of the two TC conditions, completed a structured travel request interaction, and then rated their perceived trust (PTru), autonomy (PA), transparency (PTrn), ease of use (PEU), and usefulness (PU). Our findings indicate that TC strategy had a significant overall impact on users’ combined perceptions, and that demographic factors – particularly age – played a critical role in shaping these responses, highlighting the need for adaptive, inclusive design practices that balance automation with user control. CCS Concepts • Human-centered computing → Human computer interaction (HCI) → Interactive systems and tools • Human-centered computing → Interaction design → Systems and tools for interaction design.