Fine-Tuning Large Language Models for Nonlinear Dynamics: The Lorenz Attractor as a Benchmark for Hypersonic Simulation

Traditional high-fidelity 6-DOF simulations of complex systems such as hypersonic vehicles are computationally expensive, requiring intensive numerical integration, nonlinear aerodynamics, and adaptive control computations, which presents a particular problem for real-time simulation. As an initial step toward exploring fast surrogate models for such systems, this paper investigates the use of large language models (LLMs) to emulate nonlinear, chaotic dynamics using the Lorenz attractor as a simplified benchmark. The Lorenz system, a well-known 3-state chaotic dynamical system, provides a tractable test case for assessing whether LLMs can model sensitive, time-dependent behaviors, though it represents substantially lower complexity than full aerospace dynamics. We fine-tune a foundation LLM (LLaMA 3.2 3B) using time-series data generated from the Lorenz system and evaluate its ability to learn and reproduce key dynamical features. Despite their origins in text processing, LLMs have demonstrated surprising generalization across modalities, with recent advances enabling applications such as speech synthesis from purely text-trained models. We investigate whether this emergent capability may extend to modeling nonlinear dynamical systems. Working within resource constraints (16GB VRAM, 3B parameters), systematic experimentation across multiple training phases reduced RMSE from 2.52 to 0.73 (71% improvement) compared to the base model, while maintaining 100% parse rate across 4,500 evaluation samples and achieving 8-11× inference speedup over our simulation framework's numerical integration pipeline. Valid Prediction Time (VPT) ranged from 0.11 to 4.53 Lyapunov times (mean ~0.5), with the best case comparable to Long Short-Term Memory (LSTM) network performance reported in prior work. These results position the fine-tuned LLM between naive baselines and purpose-built methods: the best-case VPT suggests transformer-based approaches are able to achieve meaningful prediction horizons, while the mean VPT falls below Echo State Networks (9-10+ Lyapunov times) achieved with far fewer parameters. Out-of-distribution testing on extreme parameter variations demonstrates 100% stability retention. Several factors, including memory-constrained inability to implement chaos-aware loss functions and use of a relatively small 3B parameter model, suggest potential for improvement with additional resources. These preliminary results warrant further investigation through scaling studies and continued benchmarking against established surrogate methods.