Paper Detail

HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

Gyeonghun Kim, Donghoon Kim, Minsoo Choi, Suyun Lee

huggingface Score 7.0

Published 2026-11-02 · First seen 2026-11-07

General AI

Abstract

Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{kim2026helm,
  title = {HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation},
  author = {Gyeonghun Kim and Donghoon Kim and Minsoo Choi and Suyun Lee},
  year = {2026},
  abstract = {Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-rele},
  url = {https://huggingface.co/papers/2611.02660},
  keywords = {Vision-Language-Action (VLA) models, Markovian assumption, non-Markovian environments, Hierarchical Explicit Language Memory (HeLM), factorized event detection, event-triggered memory updates, compositional generalization tasks, code available, huggingface daily},
  eprint = {2611.02660},
  archiveprefix = {arXiv},
}

Metadata

{}