Cardiac structure segmentation in cine-MRI using MVTF-Trans: A multi-view temporal fusion vision transformer with multi-modal feature fusion

Purpose To develop and validate a Multi-View Temporal Fusion Transformer (MVTF-Trans) that integrates spatial information from multiple cardiac views with temporal dynamics across the cardiac cycle, motivated by expert radiologist interpretation workflow, for cardiac structure segmentation. Materials and methods This retrospective multi-center study included 380 patients who underwent cine cardiac MRI at three institutions between 2021 and 2023. Data from Centers A and B (n = 285) were used for five-fold cross-validation, while Center C (n = 95) served as an independent external test set. The left ventricle, myocardium, and right ventricle were annotated across all cardiac phases. The proposed MVTF-Trans integrates view-specific Swin Transformer encoders for short-axis, two-chamber, and four-chamber views, bidirectional cross-view attention, and a temporal transformer enforcing inter-frame consistency across 20 phases, followed by a multi-scale decoder. Performance was compared with five established methods using DSC, HD95, temporal variability, and clinical indices (EDV, ESV, EF) with statistical testing and Bland–Altman analysis. Results On the external test set, MVTF-Trans achieved a mean DSC of 0.917 ± 0.024, higher than nnFormer (0.904 ± 0.026, p < 0.001). Structure-specific DSC were 0.951 (LV), 0.908 (myocardium), and 0.893 (RV), with relative gains of 1.0–1.7%. Boundary accuracy improved by 17.5% in HD95, and on the external center, MVTF-Trans maintained the highest absolute DSC (0.917) under a cross-center drop comparable to the strongest transformer baselines (≈3.1%). Temporal consistency, as measured by inter-frame Dice variability and contour smoothness, improved substantially relative to baselines; we note that these metrics are partially regularized by the temporal-consistency loss and should be interpreted alongside per-frame Dice and boundary metrics, which are not directly optimized by this loss and also showed consistent improvements. Conclusion MVTF-Trans shows that integrating multi-view anatomical reasoning with temporal modeling yields consistent improvements over single-view, single-frame baselines.