Paper Detail
Byeongho Heo, Jaehui Hwang, Sangdoo Yun, Dongyoon Han
On-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexplored. We study OPD and its advanced variant, On-Policy Delta Distillation (OPD^2), for mathematical reasoning in English, Korean, and Japanese. OPD^2 improves OPD by using the probability gap between a post-trained teacher and its base model as the learning signal. Experiments with Qwen3 show that OPD^2 consistently outperforms the original OPD, with particularly strong improvements in Korean and Japanese, and generally narrows the English-Korean performance gap. We further find that English-only OPD can also increase performance for Korean and Japanese, but often shifts the responses toward English, highlighting the importance of multilingual data to preserving target-language responses.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{heo2026policy,
title = {On-Policy Delta Distillation for Multilingual Math Reasoning},
author = {Byeongho Heo and Jaehui Hwang and Sangdoo Yun and Dongyoon Han},
year = {2026},
abstract = {On-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexplored. We study OPD and its advanced variant, On-Policy Delta Distillation (OPD\textasciicircum{}2), for mathematical reasoning in English, Korean, and Japanese. OPD\textasciicircum{}2 improves OPD by using the probability gap between a post-trained teacher and its base model as the learning signal. Experiments with Qwen3 show that OPD\textasciicircum{}2 consistently o},
url = {https://huggingface.co/papers/2608.05802},
keywords = {huggingface daily},
eprint = {2608.05802},
archiveprefix = {arXiv},
}
{}