Spatiotemporal Deformable Transformer for Lung Cancer Risk Prediction Using Low-Dose CT Images

Low-dose lung computed tomography (LDCT) has been set as the golden standard for lung cancer screening. Current state-of-the-art computer-aided diagnosis (CAD) methods aim to detect visible lung cancer from LDCT images based on well-labeled training data. To provide early alarms, we aim to predict lung cancer risk and locate the potential lung cancer region by utilizing LDCT images in two consecutive years, i.e. LDCT images of the patient captured in the first year and the second year without pixel-level labels. To achieve this goal, a novel weakly supervised spatiotemporal deformable transformer is proposed. Because LDCT images are captured at different time points without pre-alignment, our method proposes combining spatial attention maps with temporal deformable attention maps to learn spatiotemporal correlations which represent the tumor progression from LDCT images in consecutive years. To achieve weakly supervised learning of the model, we introduce three losses, including classification loss, patch token loss, and side loss. During inference, spatial attention maps with class information cooperate with spatiotemporal correlations of the test image to locate the potential lung cancer region. In the experiments, the proposed method outperforms the state-of-the-art methods and can localize the lung cancer region with only image-level labels.