Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention
arXiv:2609.24797v1 Announce Type: new Abstract: Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correction constrain their expressivity. Prior work has shown that composing two delta-rule transitions in a single recurrent update can mo