Paper Detail

Continual Adaptation Across Modalities: Learning and Forgetting in Unified Multimodal Models

Xinrui Wang, Zhihao Chen, Jia Tang, Alexandra Gomez-Villa, Bartlomiej Twardowski, Joost van de Weijer, Songcan Chen

arxiv Score 15.0

Published 2026-10-28 · First seen 2026-10-30

Research Track A · General AI

Abstract

Unified multimodal models (UMMs), which can process and generate diverse modalities within a single framework, have recently emerged as a powerful paradigm. However, when these models undergo continual learning on multiple tasks across different modalities, their learning and forgetting dynamics remain poorly understood. In this paper, we systematically investigate how architectural design and adaptation method shape intra-modal learning and cross-modal interference between the generation and understanding sides. Specifically, we report three findings. First, the severity of cross-modal interference decreases with the degree of modality-specific parameter isolation. Second, a within-model asymmetry: the generation side of a UMM is markedly more fragile than its understanding side. Third, what post-training adds is the first to be forgotten: purely post-training capabilities such as personalized generation collapse by 84\% to 90\%, while pretraining capabilities remain stable. Together, these results clarify how parameter isolation and post-training fragility shape UMMs' learning and forgetting dynamics, offering a starting point for developing continually evolving UMMs.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{wang2026continual,
  title = {Continual Adaptation Across Modalities: Learning and Forgetting in Unified Multimodal Models},
  author = {Xinrui Wang and Zhihao Chen and Jia Tang and Alexandra Gomez-Villa and Bartlomiej Twardowski and Joost van de Weijer and Songcan Chen},
  year = {2026},
  abstract = {Unified multimodal models (UMMs), which can process and generate diverse modalities within a single framework, have recently emerged as a powerful paradigm. However, when these models undergo continual learning on multiple tasks across different modalities, their learning and forgetting dynamics remain poorly understood. In this paper, we systematically investigate how architectural design and adaptation method shape intra-modal learning and cross-modal interference between the generation and un},
  url = {https://arxiv.org/abs/2610.38436},
  keywords = {cs.LG},
  eprint = {2610.38436},
  archiveprefix = {arXiv},
}

Metadata

{}