The Social Aptitude of AI: Quantifying and Assessing Driving Automation-Pedestrian Interactions

The development of human-centered Artificial Intelligence (AI) benefits from evaluation approaches grounded in human expectations. Traditional metrics (e.g., accuracy) are often insufficient for capturing how well AI systems mimic human social cognition. This study proposes a novel approach to identifying and quantifying intention prediction discrepancies between humans and AI algorithms. In Experiment I, intention predictions from 24 human participants were compared with those of two AI algorithms across 108 driving videos. We then introduced the Human-Like Index (HL-i) to measure the similarity between human and AI predictions. Experiment II validated HL-i by asking 24 participants to distinguish between human and AI predictions in human-like and non-human-like videos. Results show that AI predictions diverged from human judgments in over 50% of cases, and participants struggled to differentiate between human and AI predictions with higher HL-i scores. This method offers a meaningful tool for evaluating and improving AI social aptitude and trustworthiness.