Paper Detail

Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents

Haochen Liu, Qianhao Zhu, Zixuan Wang, Dingming Li, Tongbo Chen, Peng Qiu, Yingqi Qu, Weiming Lu, Jun Xiao, Yongliang Shen

huggingface Score 12.0

Published 2026-11-12 · First seen 2026-11-13

General AI

Abstract

Decision models answer a question with a typed output, such as a choice among options with a probability for each, so one model can choose an agent's actions, evaluate actions already taken, or score every action to teach a policy with reinforcement learning. Prior work has studied them mainly as actors, judging their usefulness by how well they act. We ask whether a decision model that acts poorly can still teach well. We compare three decision models and three general LLMs as actors and judges on ALFWorld, and use JEV as a teacher on ALFWorld and WebShop. As actors, all six models are pulled away from a subgoal that must come first by an easier one. As judges that see the whole trajectory, the same models credit the steps toward that subgoal, and asking whether each action helped, rather than whether the task will succeed, separates useful from useless actions. As a teacher, JEV raises a Qwen2.5-1.5B policy over GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop. These results suggest that a poor actor can still teach well, because judging an action after the episode ends reveals the consequences that acting must anticipate. Code is available at https://github.com/ZJU-REAL/decision-agents.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{liu2026actor,
  title = {Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents},
  author = {Haochen Liu and Qianhao Zhu and Zixuan Wang and Dingming Li and Tongbo Chen and Peng Qiu and Yingqi Qu and Weiming Lu and Jun Xiao and Yongliang Shen},
  year = {2026},
  abstract = {Decision models answer a question with a typed output, such as a choice among options with a probability for each, so one model can choose an agent's actions, evaluate actions already taken, or score every action to teach a policy with reinforcement learning. Prior work has studied them mainly as actors, judging their usefulness by how well they act. We ask whether a decision model that acts poorly can still teach well. We compare three decision models and three general LLMs as actors and judges},
  url = {https://huggingface.co/papers/2611.17167},
  keywords = {decision models, typed output, reinforcement learning, policy, actors, judges, trajectories, subgoal, general LLMs, ALFWorld, WebShop, JEV, Qwen2.5-1.5B, GRPO, code available, huggingface daily},
  eprint = {2611.17167},
  archiveprefix = {arXiv},
}

Metadata

{}