Paper Detail

VFold: Symmetry-Aware Cross-Layer Value Cache Compression

Neha Verma, Sungwon Kim, Kenton Murray, Kevin Duh

arxiv Score 8.8

Published 2026-10-08 · First seen 2026-10-09

General AI

Abstract

While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache similarities. However, most existing techniques necessitate architectural changes to LLMs and incur substantial overhead. In this work, we propose a symmetry-aware value cache merging strategy that reduces cache memory while avoiding both harmful performance degradation and architectural overhead during decoding. Furthermore, we show that this approach can be exploited alongside existing cache compression techniques, composing with high-ratio quantization or key cache pruning to reach compression ratios that neither method reaches alone, with minimal additional cost. Ultimately, our findings reveal a major source of underutilized capacity in the value cache, offering a simple yet highly effective direction for scaling context windows under memory constraints.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{verma2026vfold,
  title = {VFold: Symmetry-Aware Cross-Layer Value Cache Compression},
  author = {Neha Verma and Sungwon Kim and Kenton Murray and Kevin Duh},
  year = {2026},
  abstract = {While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache similarities. However, most existing techniques necessitate architectural changes to LLMs and incur substantial overhead. In this work, we propose a symmetry-aware value cache merging strategy that reduces cache memory while avoiding both harmful performance degradation and archit},
  url = {https://arxiv.org/abs/2610.12338},
  keywords = {cs.CL, cs.LG},
  eprint = {2610.12338},
  archiveprefix = {arXiv},
}

Metadata

{}