Uncovering algorithmic inequity: a conditional mutual information framework for detecting and mitigating hidden discrimination

Abstract Machine learning (ML) systems are increasingly embedded in automated decision-making processes across sectors such as healthcare, employment, and education. Traditional fairness metrics are typically used to assess equality across broad demographic categories but cannot capture nuanced and intersectional patterns of discrimination. Thus, this paper introduces a novel methodological framework that combines multialgorithm clustering with conditional mutual information (CMI) to detect hidden subgroup-level discrimination in ML systems. By analysing real-world datasets, we uncover statistically significant discriminatory patterns that disproportionately affect multiple marginalized individuals, particularly at the intersection of protected attributes. Additionally, we evaluate mitigation strategies, such as fairness-aware representation learning, that reduce bias while maintaining predictive accuracy. The findings have two major implications: (1) they highlight the inadequacy of surface-level fairness checks in complex sociotechnical systems, and (2) they offer actionable tools and insights for developers, managers, and policymakers seeking to audit and regulate ML technologies responsibly. Our study contributes to a growing body of work on the societal impact of algorithmic technologies and advances the methodological toolkit for equitable technology governance.