Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer, Sebastian Böck, Sepp Hochreiter
arXiv 2610.26834 53 pages Read the paper
Token-by-token KronLSTM updates serialize memory access; chunkwise execution turns the within-chunk dependencies into a Gram-based triangular solve and propagates memory only between chunks.
Recurrent linear attention needs controlled memory updates; KronLSTM separates forgetting, erasing, and writing into gates inside a standard sequence-mixer block.
Hebbian writes can bury an old association and ordinary delta updates can disturb overlapping pairs; the Kronecker Delta Rule corrects one pair’s scalar association along that pair’s rank-one direction.
When random associations cannot all be fit at once, the Kronecker Delta memory need not settle to a single state; its expected update follows least squares, while its stochastic state settles into a bounded stationary regime.