Evolving Task-Agnostic Features From Task-Specific Priors for Multi-Task Dense Prediction

Previous multi-task dense prediction methods based on a decoder-centric paradigm have achieved promising results. They primarily rely on task-agnostic features extracted from a pre-trained backbone, overlooking the importance of task-specific prior knowledge. In this paper, we propose a novel approach that integrates task priors into task-agnostic features to facilitate the development of a more refined task-specific representativeness. Specifically, we introduce a CNN-based task-prior awareness module to capture comprehensive task-specific priors. As the cornerstone of feature interaction, task-agnostic features (e.g., pretrained ViT tokens) recurrently absorb the extracted task priors, evolving into multi-task-informed features. In the meantime, multi-task-informed features enhance task-specific features in favor of dense prediction. Furthermore, to address the inherent incompatibility between ViT tokens and CNN-based priors, we incorporate multi-scale collaborative feature filters within the decoder that adaptively generate low-pass and high-pass kernels to modulate their collaboration. Extensive experiments on the PASCAL-Context and NYUD-v2 datasets demonstrate that our approach achieves comparable or even better performance than the existing state-of-the-art methods.