TORA: Parameter-Efficient Token-Driven Residual Adaptation of Remote Sensing Foundation Models for Multiscale Agricultural Vision

Precision agriculture increasingly relies on robust visual understanding of crops and fields. However, agricultural scenes remain challenging due to large appearance variability, background clutter, and fine-grained multi-scale structures. Although remote sensing foundation models (RSFMs) have demonstrated strong generalization in generic Earth observation tasks, adapting them effectively to domain-specific agricultural applications is still non-trivial. Existing parameter-efficient fine-tuning (PEFT) strategies often struggle to balance semantic adaptability across diverse crop phenotypes and domain shifts with spatial fidelity in preserving fine-grained structures and boundaries, which limits their effectiveness on dense prediction tasks. To this end, we propose TORA (Token-driven Residual Adaptation), an efficient adaptation paradigm tailored to multi-scale agricultural vision. TORA augments frozen RSFMs with hierarchical learnable semantic prototypes to explicitly model domain-specific semantic patterns and to interact with intermediate representations through token-wise attention. Meanwhile, we design an instance-aware residual path to inject adaptation signals into backbone features, thereby strengthening domain alignment while effectively mitigating the degradation of spatial details. Furthermore, we establish a comprehensive evaluation benchmark spanning three representative tasks—semantic segmentation, object detection, and image classification—across nine datasets. Extensive experiments demonstrate that TORA trains only approximately 1.5%–3.8% of the backbone parameters, substantially fewer than full fine-tuning, yet achieves performance that is competitive with or superior to strong PEFT baselines and approaches full fine-tuning on multiple datasets.