Paper Detail

Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation

Jiacheng Xu, Feng Chen, Xiuneng Xu, Bo An

Browse

Workflow Queues

arxiv Score 9.2

Published 2026-09-08 · First seen 2026-09-09

General AI

Open paper source

Abstract

Existing methods for test-time reinforcement learning (TTRL) derive rewards from answer-level self-voting on unlabeled test-time tasks with canonical answers, but this breaks down for code generation because programs cannot be compared by surface form and therefore do not directly provide a usable training signal. To make TTRL applicable to code generation, we propose probe-driven TTRL, which constructs output-free probe inputs from the problem statement, executes candidate programs on these probes, and defines a Probe Consensus Reward (PCR) from the resulting behavioral agreement. PCR provides a behavioral training signal for open-vocabulary programs, but it is not a fully reliable verifier and remains susceptible to reward hacking through spurious consensus. We therefore introduce Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which converts low PCR into conservative negative updates through rank masking and controls policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning 2026-04-21 · Score 9.3 General AI TTPO: Test-Time Policy Optimization 2026-08-27 · Score 15.3 General AI Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning 2026-06-24 · Score 11.3 General AI Chart-RL: Policy Optimization Reinforcement Learning for Enhanced Visual Reasoning in Chart Question Answering with Vision Language Models 2026-04-03 · Score 10.3 General AI Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases 2026-04-27 · Score 3.3 General AI

BibTeX

@article{xu2026entropy,
  title = {Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation},
  author = {Jiacheng Xu and Feng Chen and Xiuneng Xu and Bo An},
  year = {2026},
  abstract = {Existing methods for test-time reinforcement learning (TTRL) derive rewards from answer-level self-voting on unlabeled test-time tasks with canonical answers, but this breaks down for code generation because programs cannot be compared by surface form and therefore do not directly provide a usable training signal. To make TTRL applicable to code generation, we propose probe-driven TTRL, which constructs output-free probe inputs from the problem statement, executes candidate programs on these pro},
  url = {https://arxiv.org/abs/2609.09135},
  keywords = {cs.LG, cs.CL},
  eprint = {2609.09135},
  archiveprefix = {arXiv},
}

Metadata

{}