Cubit: Token Mixer with Kernel Ridge Regression
arXiv:2605.06501v2 Announce Type: replace-cross Abstract: Since its introduction in 2017, the Transformer has become one of the most widely adopted architectures in modern deep learning. Despite extensive efforts to improve positional encoding, attention mechanisms, and feed-forward networks, the co