A Clinically Informed Two-Stage Framework for Renal CT Report Generation

[Objective] Renal cancer is a common malignancy and an important cause of cancer-related mortality; accurate preoperative characterization is essential to guide treatment and avoid overtreatment. Computed tomography (CT) plays a central role in early detection, staging, and treatment planning. The growing volume of CT studies increases radiologists’ workload and requires specialized training to reliably interpret images and document findings. However, although radiology accounts for the majority of FDA-authorized AI/ML tools, most cleared tools target narrow tasks (e.g., detection, quantification, or triage); end-to-end automated report generation remains challenging as it requires integrating visual interpretation with clinical reasoning. Recent advances, including large language models (LLMs) and, especially vision-language models (VLMs), create new opportunities to generate structured findings and draft reports in a human-in-the-loop workflow that help reduce reporting burden, enhance diagnostic accuracy, improve efficiency, and ultimately improve patient care.[Methods] We propose a clinically informed, two-stage framework for automatic radiology report generation from single-slice renal CT images. In the first stage, we detect structured clinical features from each 2D image using a multi-task learning model. In the second stage, we generate free-text reports conditioned on both the input image and the detected clinical features, leveraging a vision-language foundation model. To assess clinical accuracy and consistency, we further extract generated clinical features from model-generated reports and compare them with expert-annotated ground truth. We used a expert-labeled dataset for training and evaluation.[Results] Experimental results on a real-world cohort suggest that incorporating clinical features improved both the quality and clinical fidelity of the generated reports. The model achieved an average AUC of 0.75 across key imaging features and a maximum METEOR score of 0.33 for report generation, supporting the viability of this approach for reducing template-driven hallucinations.[Conclusion] This preliminary study suggests the feasibility of coupling structured clinical feature detection with conditioned report generation. It offers a proof-of-concept for integrating structured prediction with free-text drafting for renal CT reporting, enhancing interpretability and clinical fidelity. The results also highlight the importance of clinically meaningful evaluation metrics when developing medical AI systems.