Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework For Generalized Referring Expression Comprehension

Read full text
Authors: Juexi Shao, S S Li, Yujian Gan, Chris Madge, Vanja Karan, Massimo Poesio
Cited by: 0
References: 25

Abstract

Dialogue-Based Generalized Referring Expression Comprehension (GREC) requires models to ground the expression and unlimited targets in complex visual scenes while resolving coreference across a long dialogue context. However, existing systems struggle under distribution shift between training and evaluation domains, a gap exacerbated by the scarcity of annotated dialogue grounding data. We address this challenge with a three-tier data-synthesis method that balances realism and controllability to produce scalable supervision for dialogue-conditioned grounding. Fine-tuning on the synthesized data yields consistent, substantial improvements over prior approaches across standard evaluation metrics.

Fields of Study & Subjects

Multimodal Machine Learning Applications Speech and dialogue systems Topic Modeling

Publication details

Journal / source
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Pages
18142–18146
Publication date
21 April 2026