Dissecting AI

All papers

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

Matan Rusanovsky, Yoav Miron, Roy Uziel, Omer Belhasin, Hao Guo, Ran Zilberstein et al.

arXiv 2608.20530 21 pages Read the paper

The short version

The problem
A fast language-model shortcut guesses several next words at once, but guesses that each seem likely can still make an awkward or incorrect sequence together. Rechecking the sequence again and again can erase the speed benefit.
The idea
LiLiCorr keeps a shortlist of likely words for each position and scores how neighboring choices fit together, using information from the whole draft. For example, it can prefer “dog” followed by “barks” over individually likely words that do not form a sensible phrase. It scores all the options in one pass, then chooses the sequence from left to right.
What it got
Compared with the original shortcut, LiLiCorr increases the number of draft words accepted by 7–19%. In comparisons with other new methods, it is fastest in 63 of 72 tested settings.

Module from this paper

LiLiCorr: Correlating Parallel Drafts

Independent slot predictions can form an incoherent draft; LiLiCorr scores candidate transitions in context in one pass, then greedily selects a coherent sequence.

speculative decodingdraft modelscandidate selectionsequence coherence