Tail Task Risk Minimization in Meta-Learning From Theoretical Advances to Practical Strategies

Meta learning is a promising paradigm in the era of large models, and task distributional robustness has become an indispensable consideration in real-world scenarios. Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement. This work contributes to more theoretical investigations and practical enhancements in the field. Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate. Under certain scenarios, we incorporate the diversity regularizer into the acquisition criteria design during active subset selection and further improve meta learners' comprehensive generalization under tail risk minimization. In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, systematically analyze the diversity regularizer's impacts, and practically improve the studied strategy. Accordingly, extensive evaluations on tasks such as few-shot sinusoid regression, system identification, image classification, and meta reinforcement learning, along with experiments on multimodal large models, demonstrate the significance, robustness and scalability of our proposal.