Paper Detail
Yaru Niu, Binghong Chen, Changwei Yao, Shangtao Li, Zhenlong Fang, Revanth Krishna Senthilkumaran, Shuai Zhou, Yuemin Mao, Kalie Ching, Po-Ju Chao, Yudong Luo, Peiqi Yu, Heng Zhang, Hao Zhang, Bingqing Chen, Chen Qiu, H. Eric Tseng, Ye Yuan, Umar Iqbal, Changliu Liu, Jonathan Francis, Ding Zhao
Generalist manipulation policies can benefit from the complementary experience of humans, real robots, and simulated agents. However, differences in morphology, sensing, and control interfaces make it difficult to learn effectively from their combined demonstrations. We introduce EgoX, an egocentric cross-embodiment learning framework that organizes heterogeneous demonstrations through semantically corresponding observation and action modalities. EgoX pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on this mixture and fine-tunes it on target-embodiment demonstrations. Within each embodiment, MXT+ assigns a dedicated tokenizer to each observation modality and a dedicated expert to each action modality, with a Transformer trunk shared across embodiments. A training-only embodiment-dreaming objective predicts latent representations of future visual and proprioceptive states, encouraging the shared trunk to model each embodiment's dynamics. Across 13 task-embodiment pairs spanning five tasks and six robot embodiments, including five physical robots and one simulated humanoid, cross-embodiment pretraining raises MXT+'s average normalized task progress from 0.63 to 0.80 over training from scratch. The gains are particularly pronounced under out-of-distribution conditions, where average task progress increases from 0.53 to 0.77. Ablations demonstrate the benefits of broader pretraining mixtures, modular interfaces, and embodiment dreaming, with dreaming producing more similar latent geometry across embodiments. Project page: egox-dream.github.io.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{niu2026egox,
title = {EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming},
author = {Yaru Niu and Binghong Chen and Changwei Yao and Shangtao Li and Zhenlong Fang and Revanth Krishna Senthilkumaran and Shuai Zhou and Yuemin Mao and Kalie Ching and Po-Ju Chao and Yudong Luo and Peiqi Yu and Heng Zhang and Hao Zhang and Bingqing Chen and Chen Qiu and H. Eric Tseng and Ye Yuan and Umar Iqbal and Changliu Liu and Jonathan Francis and Ding Zhao},
year = {2026},
abstract = {Generalist manipulation policies can benefit from the complementary experience of humans, real robots, and simulated agents. However, differences in morphology, sensing, and control interfaces make it difficult to learn effectively from their combined demonstrations. We introduce EgoX, an egocentric cross-embodiment learning framework that organizes heterogeneous demonstrations through semantically corresponding observation and action modalities. EgoX pretrains a Modularized Cross-Embodiment Tra},
url = {https://huggingface.co/papers/2611.16728},
keywords = {generalist manipulation policies, egocentric cross-embodiment learning, heterogeneous demonstrations, observation modalities, action modalities, Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+), tokenizer, expert, Transformer trunk, embodiment-dreaming objective, latent representations, visual and proprioceptive states, cross-embodiment pretraining, out-of-distribution, latent geometry, modular interfaces, code available, huggingface daily},
eprint = {2611.16728},
archiveprefix = {arXiv},
}
{}