Training variation of physically-informed deep learning models

Abstract A successful deep learning network is highly dependent not only on the training dataset, but also on the training algorithm used to condition the network for a given task. The loss function, dataset, and tuning of hyperparameters all play an essential role in training a network, yet there is not much discussion on the reliability, consistency, or reproducibility of a training algorithm. With the rise in popularity of physics-informed loss functions, this raises the question of how consistent one’s loss function is in conditioning a network to enforce a particular governing equation across different training runs. Reporting the model variation (quantified by metrics such as standard deviation across training runs) is needed to assess a loss function’s ability to consistently train a network to obey a given governing equation, and provides a fairer comparison among different methods. In this work, a Pix2Pix network predicting the stress fields of high elastic contrast composites is used as a case study. Several different loss functions enforcing stress equilibrium are implemented, and the variations in convergence, accuracy, and enforcing stress equilibrium across many training sessions, given a fixed data split for all trainings and fixed hyperparameters for each training method, are evaluated. All physics-informed loss implementations reduced the error in the stress equilibrium constraint by at least 57% and the variability of this error across 30 different training runs by at least 81%, showing the power of physics-informed losses to train networks with more consistency and accuracy. Suggested practices in reporting model variation are also shared.