Dual-level interactive learning with symptom-aware attention for automatic radiology report generation

Automatically generating radiology reports for X-ray scans holds substantial potential to assist radiologists in mitigating the time-consuming burden of report composition. Despite notable advancements, critical challenges remain: existing approaches rely solely on grid-level or region-level features, failing to emphasize their intrinsic relationship, and language decoders lack the ability to distinguish symptom-related words from non-symptom terms, leading to imprecise descriptions. Additionally, most datasets lack region annotations, limiting model training support. To address these issues, this paper proposes two key components and a novel dataset. First, a Relative Position-Augmented (RPA) module integrates relative geometric features between grid-level elements and region-level objects. This enhances the model to associate fine-grained grid details with global anatomical regions, capturing visual representations with more clinical information while reducing feature noise. Second, a BioBERT-based model extracts domain-specific linguistic context, with a Symptom-aware Attention (SA) module atop the decoder that adaptively quantifies contributions of visual and symptom-related textual features for word prediction. This ensures the model focuses on symptom-relevant terms while suppressing redundant non-symptom content during decoding process. We also build a Region-level X-Ray (RL X-Ray) dataset containing X-ray scans, diagnostic reports, and fine-grained abnormal region annotations from 8500 patients. Comprehensive experiments validate our method, with CIDEr-D scores increasing from 43.5% to 46.2% and 15.0% to 17.6% on IU-Xray datasets and RL X-Ray datasets, respectively.