Beware of optimality criteria in design of experiments: New criterion based on distance to the ideal distribution for advanced machine learning

To improve the efficiency of molecular and material development, machine learning models are constructed between experimental conditions x and experimental results y. The first candidates of x to be experimented with are determined based on design of experiments. Although various types of optimality have been developed, I first point out the problem that conventional optimality has appropriate values when the candidates of x exist in local and biased x regions. To solve this problem, a new criterion: distance to the ideal distribution (DID) is proposed. The smaller DID values, the closer the first experimental candidates are to the ideal distribution of x. Numerical simulation data analyses confirmed that, while the conventional A-, D-, E-, G-, and I- optimality criteria had appropriate values when the distribution of x was biased, the proposed DID could select the first experimental candidates that were evenly distributed over the entire region of x. Furthermore, DID was computable even when the number of x was a huge number, such as 1 million, meaning that DID can be applied to cases where x includes time-series data, such as in automatic robot experiments. In the motivating Shubert-function case, a Gaussian process regression model built from a scattered initial set achieved r 2 = 0.955, whereas a D -optimality-selected set concentrated near bounds yielded r 2 = −0.288. Here, the “ideal distribution” denotes the target marginal distributions of x (uniform by default) estimated from a large candidate pool; DID remains computable for very large candidate sets.