Adapting Video Foundation Models for Spatiotemporal Wildfire Forecasting via Cross-Modal Progressive Fine-Tuning

Wildfires pose escalating threats to ecosystems, communities, and climate systems, highlighting the urgent need for accurate, high-resolution spatiotemporal forecasting. In this work, we explore the untapped potential of video foundation models for advancing wildfire spread prediction using multi-modal satellite data. While large-scale foundation models have transformed artificial intelligence and show promise in Geospatial Artificial Intelligence (GeoAI), their direct application to domain-specific tasks like wildfire forecasting faces two major hurdles: (1) a substantial domain gap between general pretraining data (e.g., natural images and videos) and geospatial data (e.g., multi-spectral satellite imagery), and (2) limited labeled data for fine-tuning in real-world GeoAI tasks. To address these challenges, we introduce a Cross-Modal Progressive Fine-Tuning (CMPF) strategy tailored for wildfire forecasting. CMPF combines: (1) Informed cross-modal architectural alignment, leveraging video-based Transformers pretrained on spatiotemporal tasks to better capture wildfire dynamics, and (2) Progressive fine-tuning, which gradually adapts models to wildfire-specific representations through intermediate domain adaptation before task-specific tuning. We evaluate CMPF using multiple Transformer backbones, including ViT, MViTv2, and VideoMAEv2, on wildfire spread forecasting benchmarks. Our results show that video foundation models, especially when fine-tuned progressively, outperform conventional CNNs and static vision Transformers in modeling wildfire evolution. These findings also validate the effectiveness of the proposed CMPF approach for adapting general-purpose AI foundation models to complex spatiotemporal geospatial tasks.