Semi-supervised spatiotemporal segmentation of in situ transmission electron microscopy for nanoparticle dynamics

Understanding the real-time morphological evolution of nanoparticles under varying thermal and environmental conditions is crucial for revealing the mechanisms that govern their stability, growth, and functional performance in applications such as catalysis and nanomanufacturing. In-situ transmission electron microscopy provides direct, atomic-scale visualization of these dynamic processes through sequential imaging, capturing subtle transformations on a frame-by-frame basis. However, extracting reliable shape descriptors from such sequential image data remains challenging due to high noise, low contrast, inter-particle overlap, and the manual effort required for annotation. Existing segmentation methods often treat each frame independently, overlooking the temporal continuity inherent in in-situ imaging and failing to capture subtle but critical morphological transitions that underpin particle reshaping, coalescence, and structural evolution. To address these limitations, we present Swin U-Net Transformer with Temporal Convolutional Network for Segmentation (SwinTCN-Seg), a semi-supervised, spatiotemporally-aware framework that fuses transformer-based spatial encoding with temporal modeling to enable reliable analysis of morphological evolution in dynamic nanoparticle systems. Moreover, to reduce the need for dense manual labels, SwinTCN-Seg employs a pseudo-label propagation scheme that utilizes high-confidence predictions from labeled frames to guide learning on unlabeled ones, thereby uncovering transitional configurations. We validate SwinTCN-Seg on a large corpus of in-situ sequences of gold (Au) and platinum (Pt) nanoparticles imaged from 650 °C to 900 °C under vacuum and air environments. Despite being trained on only 5% of the labeled frames, the model achieves high segmentation accuracy, particularly in high-temperature regimes ( ≥ 800 °C) where conventional methods struggle to detect complex phenomena such as faceting, sintering, and fragmentation. Code and models are available at https://github.com/kaur-manpreet325/TEM-Seg .