A Scenario-Based Evaluation Framework for Smart Cockpit AI Voice Interaction Derived from Real User Contexts

Aiming at the lack of standardized evaluation metrics for advanced capabilities in AI voice interaction within smart cockpits in the era of large language models, this paper constructs a quantitative evaluation system based on real-world scenarios. Through multi-dimensional user research and the Delphi expert method, the study scientifically identifies seven core interaction scenarios and innovatively employs generative AI technology to develop a dynamic test corpus. Using the Analytic Hierarchy Process, a two-tier indicator model centered on interaction success rate and response time is established. This system was applied to evaluate seven mainstream vehicle models in real-world tests. The results indicate substantial room for improvement in overall product performance, with a prevalent imbalance between interaction success rates and response times, as well as significant variations in scenario-specific capabilities. Notably, performance was weak in scenarios requiring complex task decomposition and execution, such as travel assistant and vehicle management functions. This system addresses the technological gap in existing evaluation methods for systematically assessing AI voice interaction in scenario-based contexts.