Reinforcement-learning optimization of vortex enstrophy in two-pair cavity microfluidic flows

We develop a closed-loop “Proximal Policy Optimization (PPO)-OpenFOAM” optimization workflow to search the geometric parameters of a two-pair cavity device (a single main channel with four cavities arranged as two vertically symmetric pairs) under laminar conditions (Re=50−150). The geometry is characterized by seven tunable parameters under symmetry and manufacturability constraints. Each candidate geometry is evaluated using a two-dimensional steady incompressible laminar solver in OpenFOAM. The reward signal is constructed from the domain-averaged enstrophy (i.e., the volume average of squared vorticity magnitude) with logarithmic scaling, aiming to enhance the cavity-mouth shear layer and recirculation structures inside the cavities. Across three trainings with different random seeds, PPO sustains higher late-stage rewards than unguided random sampling, while different seeds exhibit distinct convergence paths, suggesting multiple local high-reward regions in the constrained design space. We compare a representative pair of high- and low-reward geometries: the reward increases from about 0.30 to 1.01, corresponding to an approximately fivefold gain in domain-averaged enstrophy. Flow-field comparisons show that high-reward geometries feature a narrower main channel and a more resistive outlet configuration, strengthening velocity gradients at the cavity mouth and maintaining a stable single-core recirculation. Finally, three-dimensional high-resolution computations indicate that finite wall-induced viscous effects shift the recirculation closure upstream and reduce vortex size in the downstream cavity pair; under the present conditions, clear paired Dean-type secondary vortices are not observed. These results highlight that, in computational fluid dynamics-in-the-loop scenarios with limited computational budgets, reinforcement learning with an enstrophy-based reward can generate manufacturable microfluidic vortex geometries and provide interpretable diagnostics of differences between two- and three-dimensional physics.