Cognitive-Load-Aware Multimodal Video Segmentation for Adaptive Consumer Media Systems

View at publisher Summarise with VIRAT
Authors: Wellington Ferrari da Silva, Feng Dong, Andrew K. Abel, Anil Fernando
Cited by: 0
References: 30

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The growing demand for adaptive and user-centered video experiences has created a need for understanding viewer cognitive states during media consumption. This paper introduces a lightweight, non-invasive multimodal framework for segment-level cognitive load estimation, designed to model time-varying viewer cognitive load and reveal natural low-load intervals that coincide with narrative transitions. Using synchronized audio–visual cues such as speech rate, loudness dynamics, motion intensity, and shot pacing the proposed method maps perceptual and semantic features to NASA–TLX dimensions of Mental Demand, Effort, and Temporal Demand. A reasoning-assisted fusion mechanism, driven by a large language model, estimates cognitive load continuously without requiring physiological sensors. Experiments on narrative-aligned datasets show that the model consistently detects cognitively light moments that align with human assessments of comfortable narrative pauses. In addition to ad insertion, the results support the wider application potential of cognitive-load aware segmentation for adaptive streaming, user-oriented pacing, and cognitively sensitive media interfaces in future consumer electronics.

Fields of Study & Subjects

Video Analysis and Summarization Emotion and Mood Recognition Multimedia Communication and Technology

Publication details

Journal / source
International Conference on Consumer Electronics
Pages
1–6
Publication date
3 February 2026
Language
EN