Paper Detail

What Makes a Good Move? Learning to Act from Experience in Visual Worlds

Cehao Yang, Ziyi He, Jing Yi, Jian Guo, Hui Xiong

arxiv Score 8.3

Published 2026-11-16 · First seen 2026-11-17

General AI

Abstract

Learning visual policies requires turning interaction experience into guidance for individual decisions. Trajectory outcomes obscure action quality, while unguided exploration can revisit familiar situations. We present \textsc{LightAct}, an experience-learning framework that connects skill-guided exploration with policy internalization. Interaction segments are summarized into experience entries and consolidated into a game-specific skill that guides root collection toward broader state coverage. At each root, candidate actions are compared through short rollouts under a shared frozen policy. Environment-reported score gains are averaged across continuations and normalized into relative action rewards, producing reusable supervision for reinforcement learning. Policy training uses a compact dataset of just 1,024 decision roots collected in a single source game and paired with their action rewards. Across 170 GameWorld tasks spanning 34 games, \textsc{LightAct} improves overall progress by 23.6, 29.2, and 13.0 percentage points at the 2B, 4B, and 9B scales, respectively, with our strongest model competitive with reported proprietary-model results. Ablations show broader root-state coverage with skill guidance and consistent gains across three policy optimizers. Case studies show how rollouts distinguish actions with identical immediate gains.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{yang2026what,
  title = {What Makes a Good Move? Learning to Act from Experience in Visual Worlds},
  author = {Cehao Yang and Ziyi He and Jing Yi and Jian Guo and Hui Xiong},
  year = {2026},
  abstract = {Learning visual policies requires turning interaction experience into guidance for individual decisions. Trajectory outcomes obscure action quality, while unguided exploration can revisit familiar situations. We present \textbackslash{}textsc\{LightAct\}, an experience-learning framework that connects skill-guided exploration with policy internalization. Interaction segments are summarized into experience entries and consolidated into a game-specific skill that guides root collection toward broader state coverag},
  url = {https://arxiv.org/abs/2611.21065},
  keywords = {cs.CV, cs.AI},
  eprint = {2611.21065},
  archiveprefix = {arXiv},
}

Metadata

{}