arxiv
PublishedMay 7, 2026 at 4:00 AM
Average Attention Transformers and Arithmetic Circuits
Publisher summary· verbatim
arXiv:2605.04683v1 Announce Type: cross Abstract: We analyse the computational power of transformer encoders as sequence-to-sequence functions on vectors. We show that average hard attention can be used to simulate arithmetic circuits if they are given as an input to an encoder. The circuit families
Stay posted· Newsletter
Three short emails a week: Monday, Wednesday and Friday. Top movers, new models and the most-referenced story.
Your email is used only to send The Bubble Brief. Privacy policy
Discussion
No replies yet. Be first.
Related coverage
More from ARXIV
arxivOptimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets17harxivJudging the Judges: Confidence and Disagreement in Rubrics as Rewards17harxivContent associations with answer selection and engagement in supply chain community question answering17harxivORACLE-SWE: Quantifying the Contribution of Critical Information Signals on SWE Agents17hThe Bubble Brief
MON · WED · FRIRead AI insights every Monday, Wednesday and Friday — top movers, new models, story of the week.
Your email is used only to send The Bubble Brief. Privacy policy
Originally published on arxiv ↗