Crew Workload Estimation From Analysis of Voice Data

Maintaining optimal crew performance in complex operational environments, such as aviation, is critical to safety and mission success. While numerous studies have examined individual workload through subjective and physiological measures, research addressing team-based workload assessment, particularly when crew members undertake distinct but coordinated tasks, is limited. The human voice emerges as a promising medium for assessing these workload fluctuations. This study investigates the hypothesis that voice features of crew communication, specifically pitch variability and communication frequency, can reliably indicate changes in cognitive workload. To test this, recorded crew interactions across different test scenarios are analyzed. The study explores activity analysis, probabilistic models, and neural network-based approaches, adapted from techniques used in speech emotion recognition, to estimate operator workload. Neural network-based methods are effective in recognizing emotions from scripted data, while findings from experimental tasks indicated that cognitive workload does not consistently induce emotional modulation. Pitch analysis revealed the need to account for individual voice characteristics, while vocal activity correlated well with team workload. The results suggest the potential viability of these methods as indicators of workload, particularly when trained on more robust datasets and integrated with complementary physiological signals.