ESC-GRPO: EFFICIENT REASONING FINE-TUNING VIA ESCALATING SCAFFOLD CURRICULUM AND MASKED-BOUNDARY LOSS

ESC-GRPO (Escalating Scaffold Curriculum GRPO), a method that solves both problems at once. ESC- GRPO needs only few group members instead of many and it overcomes the learning cliff by guiding the model through expert reasoning without letting it memorize. The core idea is iterative co-authoring. Instead of showing the model a complete expert solution (SFT), adding a hint to the prompt (Scaf-GRPO; [16]), or letting the model continue from a single expert prefix (BREAD; [17]), we do something different: we interleave short chunks of expert reasoning with model-generated text throughout the reasoning trace. The expert chunks are masked from the loss—the model only gets gradient on its own tokens. And those tokens are weighted by how close they are to the expert chunk, so the model learns most from the transitions where it picks up the expert’s thread. A dedicated naked group member (no scaffold at all) tests whether the model can solve the problem on its own. Training repeats on the same sample until the naked member succeeds—or a safety limit is hit. Our contributions: 1. The ESC-GRPO algorithm: reduces GRPO group size from 256+ to 3 through iterative co-authoring with masked-boundary loss and proximity-weighted gradient (Section 3). 2. Empirical validation on DeepSeek-R1-0528-Qwen3-8B (4-bit) trained on an Apple M2 Max, showing effective training across identity, exam, and math tasks in an early window of 7 samples and 46 training iterations (Section 4). 3. Mechanistic analysis showing that ESC-GRPO changes the model through a smooth, sub-degree rotation in layers 20–35 (0.09°–1.01°), while preserving all base knowledge (cos ≥0.9999) (Section 5).