MultiPC: Multitask Learning for Joint Pansharpening and Classification From Multispectral and Panchromatic Images

Multitask learning has broad applicability in remote sensing. Although MS/PAN-based pansharpening and classification tasks are highly complementary at the feature level, they are usually modeled separately. To this end, this paper proposes the multitasking parallel architecture of MultiPC to achieve collaborative optimization. To address inconsistency among tasks, MultiPC introduces the Mutually Enhanced Feature Fusion (MEFF). For differential features between tasks, it achieves feature alignment through deformable convolution and combines overlapping and cross-attention to enhance feature complementarity between tasks. MEFF simultaneously introduces a joint optimization strategy to improve the synergy and complementarity of tasks. Furthermore, we propose the reflective learning loss function to effectively perceive the learning states across various tasks. It guides the two tasks to adaptively optimize the direction based on the historically learned knowledge, thereby clarifying the learning status of each task and promoting the overall collaborative optimization. Experiments on multiple real remote sensing datasets show that MultiPC significantly outperforms existing single-task methods in both classification and pansharpening tasks. The code is open from: https://github.com/Xidian-AIGroup190726/MultiPC.