Figures and code for "Optimizing for Difference"

The GECCO Benchmark Design Challenge formulates benchmark construction as the problem of maximizing performance differences between optimizers, measured via pairwise rank distances. We approach this as a meta-optimization task and propose an automated framework that generates and refines benchmark suites using large language models (LLMs) and evolutionary search. Candidate functions are synthesized through LLM-driven evolution and evaluated directly on their ability to discriminate between five standard optimizers under a fixed evaluation budget. We further optimise the dimensionality of each problem, treating it as a key factor influencing the solver's behaviour. Our results show that this combined approach substantially increases discriminability, improving scores from approximately 2.04 to 2.4. These findings highlight that effective benchmark design requires not only diverse function structures but also careful control of problem dimensionality, and demonstrate the potential of learning-driven methods for constructing targeted benchmarking suites.