Machine Learning-Based Active Control of Supersonic Twin-Rectangular Jet Flow

We investigate the feasibility of applying reinforcement learning (RL) to the active flow control of a supersonic twin-rectangular turbulent jet for noise reduction. Large-eddy simulations (LES) of the Mach 1.5 twin jet, with a Reynolds number of one million, constitute the learning environment. Control of the jet flow is realized through plasma actuators placed within the nozzles. The environment is observed via a three-dimensional grid of near-field pressure sensors. The reward signal is provided by an additional far-field pressure sensor, which measures the overall sound pressure level (OASPL) at a location relative to the nozzles that is representative of the final checker on an aircraft carrier. To learn a control policy that seeks to minimize the OASPL, we employ proximal policy optimization. For computational tractability, we train RL agents independently on low- and intermediate-resolution meshes. The agent trained on the low-resolution mesh is able to increase its reward over 1000 episodes of training, but its learned policy does not successfully generalize to a fully-resolved LES. The agent trained on the intermediate-resolution mesh is unable to outperform a random control policy after 300 episodes of training. In the context of numerical simulations, our results demonstrate that significant hurdles remain in the application of RL to turbulent jet noise reduction, and to the control of high-dimensional, highly stochastic systems more broadly.