MESA: Plugin Meta-Modulation for Transformer-Based Cold-Start Sequential Recommendation

Sequential recommenders aim to enhance prediction accuracy by leveraging user interaction sequences, with transformer-based models showing particularly strong performance. Among them, cold-start sequential recommenders are particularly challenging because these models typically require extensive historical data to perform optimally. Some works attempt to address this issue by enhancing the adaptive ability of the sequence recommenders with meta-learning approaches. However, they are unsuitable for enhancing the popular Transformer-based sequence recommenders: MAML-based models cannot adapt the large number of parameters of Transformers, while transition-based and metric-based meta-learning models rely on unique architectures that are incompatible with Transformer-based frameworks. Also, they usually lack mechanisms to recognize and cater to multiple interests within short interaction sequences. To address these limitations, we propose MESA, a meta-modulation plugin module specifically designed to enhance the cold-start recommendation of Transformer-based sequential recommender systems. (1) We design a meta-modulation method to directly modulate the parameters in Transformer-based sequence encoders, thus enabling the model to adapt more effectively to new users in cold-start scenarios. (2) Additionally, MESA integrates the Mixture of Experts (MoE) mechanism, which refines sequence representations by utilizing multiple experts, each focusing on different aspects of user interests. This structure enhances the personalization of the recommendation by effectively handling diverse user interests within the sequences. Experiments demonstrate the effectiveness of MESA in cold-start scenarios. Our codes are available here : https://github.com/Mushroom-cat/MESA .