Towards a framework for Bias Evaluation of AI Agents in Agentic Workflows

Multi-agent AI systems, with minimal human oversight, are increasingly deployed in high-stakes domains like hiring, healthcare, and criminal justice. However, existing bias evaluation methodologies focus on isolated Large Language Model responses, failing to address sequential agent interactions where biases can propagate through decision chains undetected. Current approaches suffer from three limitations: absence of multi-stage bias tracking, inability to distinguish systematic discrimination from model non-determinism, and risk of contamination when agents infer bias evaluation intent. This study presents a novel evaluation framework incorporating demographic swapping methodology, contamination prevention architecture, and statistical analysis for multi-agent workflows. The framework employs dual presentation systems separating agent-visible content from research metadata and control scenarios to isolate bias from random variation. Evaluation across multiple models revealed unexpected preferences favouring traditionally disadvantaged groups, while 85% of apparent variation was attributable to model non-determinism rather than demographic factors. This approach advances methodological frameworks for tracking bias propagation in autonomous AI systems.