Paper Detail

DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models

Hongze Chen, Miao Pan, Lizhou Liang, Qinfeng Li, Xinyin Ma, Yongliang Shen, Weishi Wang, Daniel Dahlmeier, Xuhong Zhang, Wenqi Zhang

huggingface Score 11.5

Published 2026-10-26 · First seen 2026-10-29

General AI

Abstract

Diffusion language models (DLMs) generate text through iterative denoising, enabling bidirectional context modeling and parallel token refinement. Recent studies have applied reinforcement learning (RL) to DLMs and improved their reasoning performance. However, the iterative denoising process makes credit assignment more challenging, as final outcome rewards provide limited information about how different tokens contribute to the final output. In this paper, we propose Denoising Token Credit Assignment (DTCA), a framework for token-level credit assignment in reinforcement learning for diffusion language models. DTCA estimates token contributions by measuring the improvement in prediction confidence of remaining masked positions after token commitment and groups these signals according to part-of-speech (POS) categories. These contribution weights are used to select informative denoising timesteps and assign token-level learning signals during RL training. DTCA further calibrates these learning signals according to token-level entropy changes induced by policy updates. Experiments on LLaDA-8B-Instruct show that DTCA consistently improves mathematical reasoning performance across different generation budgets, with gains of up to 9.0 points on MATH500 and 5.3 points on GSM8K over the base model.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{chen2026dtca,
  title = {DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models},
  author = {Hongze Chen and Miao Pan and Lizhou Liang and Qinfeng Li and Xinyin Ma and Yongliang Shen and Weishi Wang and Daniel Dahlmeier and Xuhong Zhang and Wenqi Zhang},
  year = {2026},
  abstract = {Diffusion language models (DLMs) generate text through iterative denoising, enabling bidirectional context modeling and parallel token refinement. Recent studies have applied reinforcement learning (RL) to DLMs and improved their reasoning performance. However, the iterative denoising process makes credit assignment more challenging, as final outcome rewards provide limited information about how different tokens contribute to the final output. In this paper, we propose Denoising Token Credit Ass},
  url = {https://huggingface.co/papers/2610.35201},
  keywords = {diffusion language models, reinforcement learning, token-level credit assignment, denoising token credit assignment, part-of-speech categories, denoising timesteps, token-level entropy, code available, huggingface daily},
  eprint = {2610.35201},
  archiveprefix = {arXiv},
}

Metadata

{}