Feature Importance Analysis for Building an Accurate and Efficient Surrogate Model for Airfoil Performance Prediction

In this study we perform a systematic analysis of the improvement of neural network models as more independent variables or features are added to explain the target variable. In this case the target variables are airfoil force coefficients, CL, CD and CM. The features used in this study include flow conditions, such as freestream Mach, M, and airfoil angle of attack, AoA, and shape parameters that characterize the airfoil geometry. The force coefficients were computed for 1534 airfoils from the UIUC database using computational fluid dynamics for a range of M and AoA typical of what is used for computing airfoil tables for assessing wing or blade performance. The feature importance studies performed in this effort demonstrated that the most accurate model is not always obtained when all of the features are used in model training. Typically the best model was obtained when 2 - 4 of the lowest ranking features where not included. Only one of the 6 models obtained in this study---a CM model---showed that the best accuracy was achieved when all of the features were included. We also found that the most accurate model was not the most complex model, or the slowest at making a model prediction. This illustrates that systematically evaluating model features on model performance will yield the most accurate and often a more computationally efficient model. Also in this work we introduce a new error measure, error thresholding, that gives more resolution on how the accuracy of a model is improving as more features are added, and can give a clearer indication of when the best model is achieved, as compared to using traditional error measures such as R2, RMSE, or MAE.