RFAIF Framework: A Unified Approach to Internalizing AI Behavior

This repository contains a collection of four technical notes introducing and expanding upon the RFAIF framework. The RFAIF framework provides a unified method to convert external behavioral specifications—such as system prompts and Judge definitions—into internalized model behavior through post-training. Modern Large Language Models (LLMs) are often controlled externally via runtime system prompts, which can be brittle and susceptible to user pressure. RFAIF addresses this by treating behavioral alignment not as the injection of new factual knowledge, but as response-trajectory reweighting. By applying a "Generate → Judge → Internalize" loop, desirable response trajectories that previously required external prompt steering become natively accessible to the target model. This repository includes the following four papers: 1. Promptless Behavioral Alignment via RFAIF This paper investigates whether desirable response behavior induced by a runtime system prompt can be internalized into a language model using RFAIF-guided LoRA training. It demonstrates that models can learn to select desirable response trajectories without requiring the original system prompt at inference time, effectively moving from external "runtime trajectory steering" to "internalized trajectory steering". 2. Reducing Sycophancy and Unsupported Claims via RFAIF Post-Training This note applies the RFAIF principle to intellectual honesty. It shows how an honesty-oriented Judge can evaluate noisy responses and select gold data that reduces a model's tendency for blind sycophancy, false agreement, and unsupported assertions. This results in the direct internalization of anti-sycophancy behavior. 3. Judge-Defined Model Shaping via RFAIF This paper reframes the role of the AI Judge. It argues that the Judge is not merely a post-hoc evaluator, but a data-distribution shaper. By changing the natural-language Judge definition, we change the selected gold data, which in turn shapes the learned behavior of the target model. The Judge prompt thus serves as an executable post-training objective. 4. RFAIF Framework: Generate, Judge, Internalize The final paper unifies the preceding studies into a general conceptual framework. It formulates RFAIF as "behavioral specification by selection," demonstrating how diverse candidate behaviors can be evaluated by a Judge, selected as gold data, and internalized to create tailored models for enterprise alignment, persona generation, and beyond