ST-multi-ISWNet: an internal solitary wave instance segmentation algorithm based on spatial-temporal multi-modal under high sea states

Internal solitary waves are rapidly propagating disturbances that pose hazards to maritime operations while contributing to marine ecosystems through nutrient uplift. Most existing internal solitary wave detection approaches are trained and validated under low-to-moderate sea states, where surface manifestations remain visually discernible. Under high sea states, however, wind–wave clutter substantially masks internal solitary wave induced roughness modulations, resulting in persistent under-detection and biased climatological assessments. To overcome this limitation, ST-Multi-ISWNet is proposed. The principal innovation lies in the joint design of two components: first, an adaptive spectral extraction module that disentangles internal solitary wave signals from clutter in the wavenumber domain; second, a cross-attention multimodal attention mechanism that explicitly conditions instance segmentation on spatiotemporal environmental forcings. Evaluations conducted on datasets acquired under high sea states demonstrate that ST-Multi-ISWNet consistently outperforms existing segmentation models, achieving improvements of 11% in accuracy and 10% in mAP50. ST-Multi-ISWNet offers an effective framework for more comprehensive detection of internal solitary waves under high sea states. The results indicate that 24.15% of internal waves in the South China Sea during winter were underestimated in previous studies. In addition, the detection outcomes are cross-validated using SWOT data.