arxiv
PublishedMay 20, 2026 at 4:00 AM
A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
Publisher summary· verbatim
arXiv:2605.18933v1 Announce Type: new Abstract: Pre-norm Transformers with RMSNorm tolerate ternary {-1,0,+1} weight quantization with surprisingly small loss (Ma et al., 2024). We give a geometric explanation via sign-magnitude decomposition of weight perturbations. In a two-layer ReLU + RMSNorm mo
Stay posted· Newsletter
A 5-min weekly brief — top movers, price watch, story of the week.
Discussion
No replies yet. Be first.
Related coverage
More from ARXIV
arxivDemocratizing Agent Deployment Safety: A Structural Monitoring Approach18harxivHarnessing LLMs for Reliable Academic Supervision: A Comparative Study18harxivAccelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap18harxivTIDE: Trustworthy and Interpretable Battery Degradation Estimation with Contextual Learning and Symbolic Distillation18hThe Bubble Brief
WEEKLYRead AI insights every Tuesday — top movers, new releases, story of the week.
Originally published on arxiv ↗