Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection

Large Language Models (LLMs) have demonstrated strong reasoning capabilities and achieved remarkable performance on mathematical reasoning tasks.Recently, distilling reasoning ability from long-form Chain-of-Thought (CoT) has emerged as a promising approach for enhancing small-scale models.Most existing studies treat small-scale models as students and leverage long-form CoTs generated by superior teacher models as training targets for Supervised Fine-Tuning (SFT) to distill reasoning ability.However, such long-form reasoning trajectories are often misaligned with the student's capacity, making it difficult for the student model to effectively internalize the provided reasoning steps.To address this limitation, we propose errOr-aware self-ReflectION (ORION), a framework that adapts teacher CoTs through an error-aware self-reflection process.Specifically, ORION prompts the student model to refine teachergenerated trajectories by explicitly incorporating its own reasoning errors, thereby producing student-tailored reasoning trajectories for SFT.Experiments on multiple mathematical reasoning benchmarks demonstrate that ORION consistently improves student performance by more than 2% over all baselines.Further analysis shows that the CoTs constructed by ORION exhibit higher coherence and logical consistency, making them more effective supervision signals for SFT.All codes are available at https://github.com/NEUIR/ORION* indicates equal contribution.† indicates corresponding author.Question: What is the smallest positive perfect cube that is the sum of three consecutive integers?Ground-Truth: 27 …The answer is 27… Error Analysis: …misreading the condition of consecutive… Correct Reasoning: …the three integers 8,9,10 and sum 27… Vanilla Distillation ORION 𝑴 𝒔𝒕𝒖𝒅𝒆𝒏𝒕 𝑴 𝒕𝒆𝒂𝒄𝒉𝒆𝒓 𝑴 𝒔𝒕𝒖𝒅𝒆𝒏𝒕 𝑴 𝒕𝒆𝒂𝒄𝒉𝒆𝒓 Error-Aware CoT ORION incorporates errors into model-fit reasoning.The answer is 8 Error Example 1: Conclude smallest cube is 1.The final answer is boxed{1}… Error Example 2: SFT Reasoning Refinement … SFT Errors →