Metric Inversion in Generative AI Translation: Divergence Between Automated Evaluation Metrics and Human Expert Judgement in the LRM Era

The paradigm of computer-assisted translation (CAT) is currently navigating a period of profound structural disruption fueled by Generative Artificial Intelligence (GAI). The transition from deterministic Neural Machine Translation (NMT) to reasoning-driven Large Reasoning Models (LRMs), such as DeepSeek R1, has introduced complex chain-of-thought (CoT) capabilities. However, this advancement has precipitated a “Metric Inversion” crisis. Traditional automated evaluation metrics, specifically BLEU and chrF, rely on surface-level n-gram overlap against a static human reference. As AI capability shifts from literal sequence mapping to creative semantic restructuring, these legacy benchmarks systemically penalize the exact linguistic and idiomatic qualities that human experts value most. Through a randomized double-blind study (N = 111), we contrast the performance of an LRM, a general LLM, and a standard NMT system. Results demonstrate that while human experts strongly favor LRM outputs for their superior fluency (Mean=8.81) and idiomaticity (Mean=8.61), automated metrics exhibit a significant and misleading bias toward traditional NMT (BLEU=0.258). Correlation analysis reveals a profound negative relationship (ρ = −0.401, p