Multi-scale Cross Vision Transformer Framework for Nutritional Value Estimation in Food Images

Estimating nutritional values from food images is a complex task requiring the integration of fine-grained visual textures and global contextual cues related to portion structure and ingredient composition.Conventional convolutional approaches frequently underperform in this setting, while multimodal or depth-augmented systems impose practical constraints on scalability and data acquisition.To address these limitations, we propose a Multi-Scale Cross Vision Transformer (Cross-ViT) framework specifically adapted for nutritional value regression from RGB images.The method incorporates dual-scale patch embeddings and a bidirectional cross-attention mechanism that facilitates interaction between localized texture patterns and global dish-level structure.A nutrient-specific multi-head fusion module is further introduced to enhance discriminative representation for calorie, protein, fat and carbohydrate estimation.Evaluated on the Nutrition5k benchmark, the proposed framework achieves a Mean Absolute Error (MAE) of 28 kcal and a Mean Absolute Percentage Error (MAPE) of 13.5% for calorie prediction, outperforming CNN-based models and surpassing several depth-augmented methods despite using only RGB inputs.Nutrient-wise performance demonstrates MAE values of 6 g (protein), 5 g (fat), and 10 g (carbohydrates), with corresponding MAPE improvements ranging from 1.5-4% over prior state-of-the-art.Ablation experiments highlight the critical contributions of cross-attention (reducing average error ~3%) and nutrient-aware fusion.Qualitative attention-map analysis further confirms meaningful alignment with nutritionally salient regions.These results underscore the effectiveness of multi-scale transformer architectures for advancing automated nutrition estimation in food image analysis.