AI Has Two Memory Problems. We're Only Talking About One.
The Breakthrough Everyone’s Talking About Two weeks ago, Moonshot AI’s Kimi team published Attention Residuals (arXiv:2603.15031) — a fundamental redesign of how information flows through transformer layers. The results are striking: 7.5-point improvement on science reasoning, 1.25× compute efficiency, and the theoretical ability to stack infinite layers without signal collapse. The core insight is elegant. Standard transformers use fixed residual connections — each layer adds its output to a running sum, like throwing every ingredient into one pot. By the time you reach layer 100, the signal from layer 3 is buried under an avalanche of accumulated noise. ...