Improving Attitude and Heading Reference Systems Performance via Machine Learning-driven Parameters Fine-Tuning

Attitude and Heading Reference Systems (AHRSs) based on Error-State Extended Kalman Filters (ES-EKF) require careful tuning of stochastic parameters to achieve optimal performance. Traditional approaches rely on Allan Variance (AV) analysis for parameter identification, which, while physically grounded, does not guarantee optimal navigation performance. This paper presents three advanced optimization methodologies—Gaussian Processes (GP), Nondominated Sorting Genetic Algorithm III (NSGA-III), and Multi-Objective Tree-structured Parzen Estimator (MO-TPE)—to systematically fine-tune critical ES-EKF parameters directly against flight performance metrics. The optimization targets accelerometer/magnetometer noise/bias characteristics, vehicle dynamics/Earth magnetic field uncertainty, and Innovation Filter (IF) thresholds across a multi-dimensional parameter space. Experimental validation using 19 real Unmanned Aerial Vehicle (UAV) flights demonstrates substantial improvements over traditional AV-based tuning. The proposed methods achieve 8191% reductions in Root Mean Square (RMS) attitude errors, 82-91% improvements in Mean Absolute Errors (MAE), and 41-93% enhancements in estimation consistency across roll, pitch, and yaw axes. MO-TPE consistently delivers the best overall performance, achieving optimal results in 7 out of 9 metric-axis combinations, followed closely by NSGA-III, while GP provides competitive single-objective optimization. The largest improvements are observed in yaw estimation, where traditional approaches struggle most with magnetic disturbances. These results demonstrate that machine learning-driven parameter optimization can significantly enhance AHRS accuracy and robustness without modifying the underlying ES-EKF structure, offering a practical path for improving navigation performance in real-world UAV applications.