Harmonizing Supervised Fine-Tuning and Reinforcement Learning with Reward-Based Sampling for Continual Machine Unlearning

Large language models (LLMs) are pretrained on massive internet data and inevitably memorize sensitive or copyrighted content. This continually raises privacy, legal, and security concerns. Machine unlearning has been proposed as an approach to remove the influence of undesired data while maintaining model utility. However, in real-world scenarios, unlearning requests continuously emerge, and existing approaches often struggle to handle these sequential requests, leading to utility degradation. To address this challenge, we propose the harmonization of Supervised fine-tuning and Reinforcement learning with Reward-based Sampling (SRRS) framework, which dynamically harmonizes supervised fine-tuning (SFT) and reinforcement learning (RL) via reward signals: SFT ensures forgetting efficacy, while RL preserves utility under continual adaptation. By harmonizing these paradigms, SRRS achieves reliable forgetting and sustained utility across sequential unlearning tasks, demonstrating competitive performance compared to baseline methods on TOFU and R-TOFU datasets.