Paper Detail
Zhaoyi Li, Deyang Kong, Yuan Wei, Evan Yang, Ranran Shen, Mahardika Krisna Ihsani, Ming Yang, Wei Zhang, Chuan Hao, Jian Yang, Ran Tao, Bryan Dai, Shikun Zhang, Wei Ye, Ying Wei, Defu Lian
On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data. We present a controlled study that varies one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting. We find that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful. Transfer depends strongly on the origin relationship between teacher and student: same-origin pairs bring the student close to the teacher across languages, reasoning horizons, and even other domains, whereas cross-origin pairs mostly fit the trained distribution. This broad reach is a double-edged sword: since routing prompts to domain experts cannot confine each teacher's influence, combining them yields a mixture-dependent seesaw among their capabilities. These results clarify when OPD generalizes and offer a useful perspective for diagnosing multi-teacher OPD.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{li2026every,
title = {Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models},
author = {Zhaoyi Li and Deyang Kong and Yuan Wei and Evan Yang and Ranran Shen and Mahardika Krisna Ihsani and Ming Yang and Wei Zhang and Chuan Hao and Jian Yang and Ran Tao and Bryan Dai and Shikun Zhang and Wei Ye and Ying Wei and Defu Lian},
year = {2026},
abstract = {On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data. We present a controlled study that varies one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting. We find that OPD transfers a teacher's reasoning b},
url = {https://huggingface.co/papers/2608.16647},
keywords = {on-policy distillation, reasoning behavior, cross-domain transfer, multi-teacher, same-origin, cross-origin, distribution shift, huggingface daily},
eprint = {2608.16647},
archiveprefix = {arXiv},
}
{}