Evolutionary Contrastive Ensemble With Conditional Redundancy Fitness Evaluation for Goal-Conditioned Humanoid Locomotion

Goal-conditioned humanoid locomotion in reinforcement learning (RL) remains challenging due to sparse reward signals and the single-goal overfitting problem. Although contrastive reinforcement learning (CRL) has achieved considerable success in this setting, it can suffer from pronounced estimation variance, since epistemic uncertainty is difficult to reduce given limited task-specific information and model capacity. Ensemble-based critics can partially alleviate this issue. However, sufficient ensemble diversity and accurate individual estimates are not necessarily guaranteed during training, resulting in unstructured exploration. To address these challenges, we propose Conditional Redundancy-Guided Evolutionary Contrastive Ensemble with Direct Preference Optimization weighting (CRECE-DPO), which augments CRL with a vectorized critic ensemble and refines the ensemble via an evolutionary algorithm guided by a tailored fitness metric. Specifically, we design a DPO-weighted conditional redundancy fitness score, to prune redundant representations while promoting effective exploration of the parameter space. Simulation results on challenging goal-conditioned benchmarks, including humanoid locomotion, demonstrate consistent improvements over CRL and other baselines.