arxiv
PublishedOctober 22, 2026 at 4:00 AM
Kinetic theory for Transformers and the lost-in-the-middle phenomenon
Publisher summary· verbatim
arXiv:2605.09213v2 Announce Type: replace-cross Abstract: We study causal self-attention dynamics -- a toy model for decoder Transformers -- which we interpret as a non-exchangeable interacting particle system. Adapting cumulant expansions to the triangular causal dependency structure of the model,
Stay posted· Newsletter
Three short emails a week: Monday, Wednesday and Friday. Top movers, new models and the most-referenced story.
Your email is used only to send The Bubble Brief. Privacy policy
Discussion
No replies yet. Be first.
Related coverage
More from ARXIV
arxivHow to Use Parallel Data for Multilingual Pretraining5harxivKIRA: A Full-Stack Reading Screening System with Dual-Pathway Analysis for Children5harxivBuilding AI Research Teams for Autonomous Discovery of Linear Attention Architectures5harxivCSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer5hThe Bubble Brief
MON · WED · FRIRead AI insights every Monday, Wednesday and Friday — top movers, new models, story of the week.
Your email is used only to send The Bubble Brief. Privacy policy
Originally published on arxiv ↗