Paper Detail

SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering

Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao

huggingface Score 10.0

Published 2026-11-10 · First seen 2026-11-12

General AI

Abstract

Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce SteerDistill, a quantization-aware distillation method in which a full-precision teacher guides the student to revise failed responses and iteratively refines its guidance using feedback from student execution. This closed-loop teacher steering produces locally verified behavioral corrections. We internalize these corrections by training the student without guidance to match the steered student's predictions at corrected turns and the full-precision teacher's at other turns, enabling unassisted inference. Across five agentic benchmarks and three models, SteerDistill exceeds the strongest quantized baseline by 1.67--2.18 points in average score under W4A16 and stays within one point of BF16. On τ^2-Bench, it outperforms QeRL by 1.84 percentage points in success rate using 67% fewer training GPU hours and produces shorter successful trajectories than the quantized baselines.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{cui2026steerdistill,
  title = {SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering},
  author = {Genming Cui and Xiangyu Li and Zixu Hao and Xin Ding and Yunxin Liu and Ting Cao},
  year = {2026},
  abstract = {Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce },
  url = {https://huggingface.co/papers/2611.13410},
  keywords = {low-bit quantization, LLM agents, quantization-aware distillation, full-precision teacher, student model, rollouts, environment transitions, full-precision references, closed-loop teacher steering, behavioral corrections, unassisted inference, W4A16, BF16, agentic benchmarks, τ\textasciicircum{}2-Bench, QeRL, code available, huggingface daily},
  eprint = {2611.13410},
  archiveprefix = {arXiv},
}

Metadata

{}