Synaesthesia in digital multimodal composing: the case of a mobile-assisted task for learning Chinese as a foreign language

Evaluating creativity in digital multimodal composing (DMC) remains a persistent challenge in second language pedagogy, as teachers often rely on intuition rather than principled frameworks. This study addresses this gap by investigating the dynamics of DMC in the context of learning Chinese as a foreign language, exploring how learners employ visual and verbal resources to construct meaning creatively and engage audiences. Grounded in a synaesthetic theoretical perspective and informed by the grammar of visual design, it analyses student-generated artefacts and interviews from beginning-level learners at an Australian university who completed a WeChat-based mobile-assisted task. Synaesthetic analysis identified three mechanisms of intermodal interaction—synaesthetic representation, interaction, and composition—that underpin creative meaning-making in DMC. It also revealed key challenges, including simplified intermodal relations, partial investment in design, and underdeveloped intermodal orchestration. While the benefits of DMC for language learning are well recognised, the study highlights the need for clearer frameworks to evaluate intermodal creativity through a synaesthetic lens. The research contributes theoretically by introducing a structured synaesthesia-informed framework for analysing intermodal relations, thereby extending multimodal scholarship in computer-assisted language learning. Practically, it offers concrete pedagogical guidelines for assessing and fostering creativity in DMC tasks through multimodal feedback, learner agency, and flexible assessment design.