Who Is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition

There is increasing interest in applying artificial intelligence (AI) to automate and support complex decision-making tasks. However, it remains unclear how algorithms compare to human judgment in contexts requiring semantic understanding and domain expertise. We examine this in the context of the judge assignment problem --- matching submissions to suitably qualified evaluators --- at a prominent U.S. university startup competition. Awarding over $\textdollar$500,000 annually, this is a real-world setting where high-quality judge assignment is critical. We develop and deploy HLSE (Hybrid Lexical–Semantic Similarity Ensemble), an AI-based approach, at the competition and compare algorithmic against human expert assignments by collecting blinded match quality scores from judges for $309$ judge-venture matches. Using a Mann–Whitney U statistic based test, we found no statistically significant difference in assignment quality between the two approaches ($AUC=0.48, p=0.40$). On average, algorithmic matches are rated $3.90$ and manual matches are rated $3.94$ on a $5$-point scale, where $5$ indicates an excellent match. Furthermore, manual assignments that took a full week in past years can be completed in under ten minutes by the algorithm during deployment. These results demonstrate that HLSE achieves human expert level matching quality while offering greater scalability and efficiency, underscoring the potential of AI-driven solutions to robustly support and enhance human decision-making for judge assignment in high-stakes settings.