Multi-Group Fairness Measures for Classification

This paper proposes a robust, reliable, and reproducible methodology for evaluating fairness in classification algorithms. Building upon established theoretical definitions of non-discrimination criteria—Independence, Separation, and Sufficiency—we present a comprehensive approach to quantifying fairness. Notably, our methodology extends beyond binary group comparisons to accommodate scenarios with multiple sensitive groups. Furthermore, we introduce a novel method for assessing fairness under the Sufficiency criterion, specifically focusing on Calibration. We assess the reliability and robustness of our model through applications to two real-world datasets.