Knowledge-Augmented Adaptive Mechanism for Radiology Report Generation

Radiology report generation, which aims to relieve the heavy workload of radiologists and reduce the risks of misdiagnosis and overlooked diagnoses, is of great significance in current clinical medicine. Most existing methods mainly formulate radiology report generation as a problem similar to image captioning. Nevertheless, in the medical domain, these data-driven methods are plagued by two key issues: the insufficient utilization of expert knowledge and visual–textual biases. To solve these problems, this study presents a novel knowledge-augmented adaptive mechanism (KAM) for radiology report generation. In detail, our KAM first introduces two distinct types of medical knowledge: prior knowledge, which is input-independent and reflects the accumulated expertise of radiologists, and posterior knowledge, which is input-dependent and mimics the process of identifying abnormalities, thereby mitigating the issue of visual–textual bias. To optimize the utilization of both types of knowledge, this study develops a knowledge-augmented adaptive mechanism, which integrates the visual characteristics of radiological images with prior and posterior knowledge into the decoding process. Experimental evaluations on the publicly accessible IU X-ray and MIMIC-CXR datasets indicate that our approach is on par with the current common methods.