Validation of Dynamic Bayesian Optimization for a Non-Stationary Human-in-The-Loop Optimization Problem

Human-in-the-Loop Optimization (HILO) has demonstrated efficacy in achieving a plethora of assistive or augmentative effects. However, conventional optimizers such as Bayesian optimization (BO) do not account for the non-stationary aspects of the human-robot system, and may thus be limited in the domains of robot-assisted training or rehabilitation. In this study, we implemented HILO using dynamic Bayesian optimization (DBO) to define the optimal value of a single control parameter to target a desired effect in propulsion mechanics, specifically in the maximum hip extension (HE) angle during stance. Sixteen healthy participants received unilateral hip torque pulses via a hip exoskeleton for about 15 minutes while walking on a treadmill. Exoskeleton torques were determined via HILO using DBO or BO. Unbeknownst to the optimizers, treadmill speed was gradually increased to amplify the non-stationary behavior of the system. Experimental results revealed that DBO outperformed BO in the later stages of training, leading to improved cost and to final values of torque closer to the true value. The primary difference between optimizers arose from their ability to model the history-dependent relationship between applied torque and output. BO associated later deviations between expected and measured output to process noise, while DBO developed a more accurate predictive model of the input-output relationship that gave proper weighing to past datapoints for making predictions.