arxiv
PublishedOctober 19, 2026 at 4:00 AM
Where Prefill Expert Reduction Hurts MoE Models: A Few Prompt Tokens Carry Most of the Short-Answer F1 Loss
Publisher summary· verbatim
arXiv:2610.22454v1 Announce Type: new Abstract: Running prompt tokens through fewer routed experts during prefill is a simple way to make mixture-of-experts (MoE) serving cheaper, yet it is unclear what this damages. We locate the damage with token-, layer- and attention-path interventions on short-
Stay posted· Newsletter
Three short emails a week: Monday, Wednesday and Friday. Top movers, new models and the most-referenced story.
Your email is used only to send The Bubble Brief. Privacy policy
Discussion
No replies yet. Be first.
Related coverage
More from ARXIV
arxivAn AI Security Agent for University ACMIS: Multi-Vector Threat Detection and Automated Response7harxivLearning Transferable Latent User Preferences for Human-Aligned Decision Making7harxivRepair the Core, Not the Symptom: Planning-Graph Repair for Long-Horizon Agents7harxivIndustryForge-27B: A Domain-Enhanced Multimodal Foundation Model for Industrial CAD7hThe Bubble Brief
MON · WED · FRIRead AI insights every Monday, Wednesday and Friday — top movers, new models, story of the week.
Your email is used only to send The Bubble Brief. Privacy policy
Originally published on arxiv ↗