Paper Detail

Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos

Wendong Bu, Tinghong Ye, Qizhou Wang, Jiacheng Li, Zhiqi Ge, Yuze Lin, Zhongqi Yue, Yi Su, Jingyu Wang, Kai Shen, Jingdong Liu, Chengyuan Deng, Xiaoxuan Chen, Xin Tan, Jiaming Ji, Wenqiao Zhang, Chenghong Jiang, Tuoyu Li, Siliang Tang, Jun Xiao, Tat-Seng Chua, Yueting Zhuang, Xingxing Wang, Juncheng Li

arxiv Score 9.3

Published 2026-10-30 · First seen 2026-11-02

General AI

Abstract

Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal inputs while supporting real-time closed-loop control. We present Movens-5M, constructed from raw Internet videos through clip generation, quality filtering, semantic annotation, motion recovery, and physical refinement with humanoid retargeting and execution verification. It contains 15 million human-motion sequences and 5 million physically executable humanoid-motion sequences spanning over 10,000 hours, aligned with text, RGB video, speech, and music. Building on this data, we introduce Movens-Zero, a scalable understanding-generation-execution architecture that supports joint learning across modalities and real-time closed-loop execution. Task-mixed flow-matching training jointly optimizes a single action expert across seven generation tasks, enabling direct motion generation from text, speech, music, and video. Multimodal Real-Time Chunking connects overlapping chunks under continuously updated instructions and robot states for continuous closed-loop execution. We further introduce Movens-Intent, a four-track benchmark with 8,192 expert-verified condition-motion pairs across text, speech, music, and video. Experiments show consistent gains across all four modalities when scaling execution-verified data from 1% to 100% and action-expert capacity from 64M to 911M parameters, while demonstrating that data quality and cross-modal supervision improve generation and execution.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{bu2026movens,
  title = {Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos},
  author = {Wendong Bu and Tinghong Ye and Qizhou Wang and Jiacheng Li and Zhiqi Ge and Yuze Lin and Zhongqi Yue and Yi Su and Jingyu Wang and Kai Shen and Jingdong Liu and Chengyuan Deng and Xiaoxuan Chen and Xin Tan and Jiaming Ji and Wenqiao Zhang and Chenghong Jiang and Tuoyu Li and Siliang Tang and Jun Xiao and Tat-Seng Chua and Yueting Zhuang and Xingxing Wang and Juncheng Li},
  year = {2026},
  abstract = {Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal inputs while supporting real-time closed-loop control. We present Movens-5M, constructed from raw Internet videos through clip generation, quality filtering, semantic annotation, motion recovery, and physical refinement with humanoid retargeting and execution verification. It contains 15 million human-motion sequences a},
  url = {https://arxiv.org/abs/2610.41877},
  keywords = {cs.RO, cs.CV},
  eprint = {2610.41877},
  archiveprefix = {arXiv},
}

Metadata

{}