Pseudo-Likelihood Training for Reasoning Diffusion Language Models

Read full text Summarise with VIRAT
Authors: Shiv Shankar Shankar
Cited by: 0
References: 48

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Policy-gradient reinforcement learning (PGRL) forms the backbone of current methods used to enhance alignment and reasoning in Large Language Models (LLMs).However, these methods are incompatible with diffusion based language models (dLLMs).Most attempts to apply PGRL to dLLMs, are either not scalable or use unprincipled approximations.This work, introduces PADRE a framework that uses a novel pseudo-likelihood based objective for alignment of dLLMs.Our objective has the same optima as PGRL based optimization, but does not need to evaluate exact likelihood from dLLMs.Experiments on various coding and mathematical reasoning benchmarks show that our method matches or surpasses the performance of recent dLLM training baselines such as diffu-GRPO/d1.Our approach provides a stable and practical alternative for RL-based fine-tuning of reasoning-focused dLLMs.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Reinforcement Learning in Robotics

Publication details

Journal / source
Conference of the European Chapter of the Association for Computational Linguistics
Pages
5514–5529
Publication date
1 January 2026
Language
EN