Multimodal analysis of spontaneous speech for predicting food liking: Integrating linguistic and prosodic features with machine learning

Spontaneous speech offers a promising yet underexplored window into consumers' emotional experiences during food evaluation. Unlike traditional self-reported measures, vocal expression captures both explicit linguistic content and implicit affective cues, potentially providing a richer account of hedonic perception. This study investigates whether linguistic and prosodic cues embedded in natural spoken responses can predict product liking. Ninety participants evaluated three chocolates and three plant-based beverages, providing hedonic ratings and open-ended spoken descriptions. Speech was analysed using psycholinguistic, prosodic, and representation-based features. Multiple machine learning algorithms were applied for both classification (low/medium vs high liking) and regression (continuous liking prediction). Feature-level comparisons showed that chocolates elicited richer emotional language, higher vocal intensity, and greater acoustic variability than plant-based drinks (p < 0.005). Across the full dataset, linguistic markers were the strongest predictors of liking (F1-score = 0.695; mean absolute error (MAE) = 1.530), while prosodic features provided complementary information related to arousal and engagement. Speech embeddings showed limited generalisation (area under the curve (AUC) < 0.518; MAE > 1.602). Product-specific analyses revealed higher predictive accuracy for chocolates (F1-score = 0.825; MAE = 1.213) than for plant-based beverages (F1-score = 0.559; MAE = 1.619), indicating that the strength of affective cues in speech depends on the emotional resonance of the product being evaluated. These findings demonstrate that spontaneous speech contains measurable emotional signals related to food liking. Interpretable linguistic and prosodic features can partially predict hedonic responses, highlighting the potential of multimodal, speech-based emotion analysis as a complementary tool in sensory and consumer research.