Modeling High-Reynolds Number Flows Using Transformers and Multi-Scale Latent Spaces

Machine learning-based models in fluid dynamics offer promise for predicting complex fluid flows at a reduced computational cost. However, existing approaches have not sufficiently demonstrated the capability to scale to problems of practical interest. This work is motivated by this gap and explores the combined application of a transformer architecture with a novel multi-scaling technique for constructing a latent space of turbulent flows. Potential towards application to problems of practical interest is assessed by first considering a two-dimensional lid-driven cavity at Reynolds numbers of 15,000 to 30,000. Despite the two-dimensional aspects of the flow, this target problem is noteworthy due to consideration of Reynolds numbers of O(10^4); providing important insights into capabilities for sufficiently predicting highly nonlinear, multi-scale fluid dynamics. Performance is benchmarked against three different models: 1) Direct Numerical Simulation, 2) a transformer architecture with a latent space constructed using Proper Orthogonal Decomposition, and 3) a simple frequency-based extrapolation of the flow field dynamics. The multi-scale transformer architecture is found to reasonably reproduce the fluid dynamics at both Reynolds numbers, whereas models using a Proper Orthogonal Decomposition latent space perform poorly. However, good agreement of a basic frequency-based extrapolation with Direct Numerical Simulation indicates that more challenging flow field dynamics are required to meaningfully stress the multi-scale transformer framework.