Paper Detail

Learning What Pairs Miss for Higher-Order Multimodal Alignment

Kunyu Zhang, Shenghui Feng, Xiangjie Shi, Zeqing Wang, Yingxu Wang

arxiv Score 11.3

Published 2026-11-16 · First seen 2026-11-17

General AI

Abstract

Contrastive learning beyond two modalities now scores the whole tuple with multilinear products, volumes, and fusion heads, critics that can represent the joint distribution. In practice these critics still behave like bags of pairs. Their negatives break at least two of the three pairs, a single pairwise check rejects them, and when most samples can be solved by pairs the higher-order term receives almost no gradient. To solve this problem, we propose CHORD (Contrastive Higher-Order Residuals with Decoys), which trains a residual against decoys drawn from a calibrated pairwise model, negatives that the pairwise model finds plausible, so that rejecting them often needs all three modalities. Drawing decoys costs one inner-product query over a memory bank, and the optimal residual is the log-ratio between the data and the pairwise model, so the decoy loss also bounds what that model misses. On M3-Mix, where one in ten training samples needs all three modalities, our CHORD improves accuracy on those samples by 21.2 percentage points over the strongest of eight baselines. CHORD also gives the best two-to-one results on every real dataset, and its residual improves other retrievers without retraining.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{zhang2026learning,
  title = {Learning What Pairs Miss for Higher-Order Multimodal Alignment},
  author = {Kunyu Zhang and Shenghui Feng and Xiangjie Shi and Zeqing Wang and Yingxu Wang},
  year = {2026},
  abstract = {Contrastive learning beyond two modalities now scores the whole tuple with multilinear products, volumes, and fusion heads, critics that can represent the joint distribution. In practice these critics still behave like bags of pairs. Their negatives break at least two of the three pairs, a single pairwise check rejects them, and when most samples can be solved by pairs the higher-order term receives almost no gradient. To solve this problem, we propose CHORD (Contrastive Higher-Order Residuals w},
  url = {https://arxiv.org/abs/2611.21372},
  keywords = {cs.LG},
  eprint = {2611.21372},
  archiveprefix = {arXiv},
}

Metadata

{}