Game-Based Robust Machine Learning under Distribution and Group Shift

This technical report presents a game-based robust learning framework for nonlinear regression under distributional heterogeneity and group shift. Robustness in machine learning is typically studied either through analytic guarantees in restricted model classes or through empirical adversarial training in highly expressive models. This work operates in the second regime. Rather than seeking closed-form certification, it investigates robustness as an empirical phenomenon emerging from structured adversarial interaction under controlled optimization. The proposed framework formulates training as a multi-player minimax game between a learner and five interacting adversaries acting at distinct levels: Sample-level reweighting within environment–group cells Group-level worst-case emphasis Environment-level stress weighting Feature-level perturbations via bounded projected gradient descent Distribution-level interpolation under controlled shift Each adversary operates under explicit budget constraints (capped simplex or bounded-norm sets) to prevent pathological dominance and ensure stable optimization. The learner is implemented as a nonlinear regression model using Random Fourier Features (RFF), balancing expressive capacity with CPU-tractable training. Optimization proceeds via alternating minimization–maximization updates with projected and mirror-style steps. Evaluation is conducted under true nested cross-validation (2 repeats × 3 outer folds) with bounded random-search tuning to prevent information leakage. Robustness is assessed using: Overall RMSE Worst-group RMSE Worst-environment RMSE CV+ (fold-jackknife+) conformal prediction Group-conditional (Mondrian) coverage Confidence intervals over outer runs Across repeated outer folds, the framework demonstrates stable worst-group and worst-environment performance, along with conservative and well-calibrated uncertainty estimates (mean conformal coverage ≈ 0.943 for nominal 0.90). No fold exhibits catastrophic degradation under worst-case criteria, suggesting robustness is systematic rather than incidental. This work does not provide analytic robustness guarantees. Instead, it characterizes robustness behavior empirically under bounded adversarial design and rigorous validation. It complements certified linear approaches by examining expressive nonlinear regimes where formal certification is unavailable. All evaluation artifacts, summary metrics, and serialized experimental results are released publicly to support full reproducibility. The contribution is methodological: it demonstrates how bounded adversarial modeling, nested validation, and distribution-free uncertainty calibration can be integrated into a unified robustness-oriented learning framework.