Recurrent Looped Transformer

Yifan Zhang, Jichen Feng, Shihan Qin, Sun-Yuan Kung, Mengdi Wang

arXiv 2610.07591 57 pages Read the paper

Modules from this paper

Chunked Feedback for Parallel Token Processing

Recurrent decoder feedback limits parallel processing across known tokens; RLT-2 holds feedback fixed within chunks and commits the final output at complete chunk boundaries.

transformerssequence-modelsattentionparallelism

Recurrent Decoder Feedback Over Parallel Encoder Memory

Fixed-depth Transformers struggle to carry algorithmic state across long inputs; RLT adds a recurrent decoder update that reuses causal encoder memory while feeding each token the previous decoder output.

transformersrecurrenceattentionsequence-modeling