Capability–Alignment Tradeoffs and the Limits of Post-Hoc Safety in Large Language Models

Large language models exhibit persistent tension between capability and alignment: post-training methods improve instruction-following and safety while often degrading benchmark performance. Across major model families (GPT-4, Claude, Llama, Gemini), evidence demonstrates three phenomena: (A) alignment-induced capability tradeoffs ("alignment tax"), (B) shallow, modular safety fine-tuning that can be removed with minimal data, and (C) high-success adversarial prompting that bypasses refusal behavior. While jailbreak effectiveness reveals the fragility of refusal mechanisms, it does not by itself prove suppressed internal knowledge exists. These findings indicate current post-hoc alignment implements safety as a fragile output constraint rather than an integrated architectural property, motivating exploration of architectures that integrate capability and safety constraints during generation.