arxiv
PublishedSeptember 3, 2026 at 4:00 AM
HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models
Publisher summary· verbatim
arXiv:2609.02029v1 Announce Type: new Abstract: Long-context inference retains a growing key--value (KV) cache during decoding, which consumes substantial GPU memory and can reduce generation throughput. This bottleneck remains in hybrid language models because their residual global-attention layers
Stay posted· Newsletter
A 5-min weekly brief — top movers, price watch, story of the week.
Discussion
No replies yet. Be first.
Related coverage
More from ARXIV
arxivMeta-ethics and AI: exploring the novel meta-ethical questions in the era of AI1harxivSSAKG 2.0: An Open-Source Package for Structural Associative Sequence Memory and Context-Based Retrieval1harxivEpistemic Sybil Resistance: Multiplying AI Agents Without Multiplying Evidence1harxivDocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents1hThe Bubble Brief
WEEKLYRead AI insights every Tuesday — top movers, new releases, story of the week.
Originally published on arxiv ↗