LiLiCorr: Correlating Parallel Drafts

Why People Wanted It

A parallel drafter can predict several positions at once, but choosing the most likely token at each position independently can combine locally plausible choices into a sequence that does not fit together. The challenge is that a block of $B$ positions over a vocabulary of size $V$ has $V^B$ possible sequences: scoring every complete block directly is not practical. Iterative correction asks for more network passes; tree reranking shifts work to the target model. So the real question is whether you can recover useful relationships between neighboring choices without explicitly modeling every whole sequence.

The Core Move

LiLiCorr gives every shortlisted candidate a context-aware incoming and outgoing vector, scores adjacent candidates by cosine similarity, and computes those vectors for all slots and candidates in one pass. Then a left-to-right greedy walk chooses a path through the candidate lattice.

For candidate $a$ at one slot and candidate $b$ at the next, the compatibility is

$s(a,b)=\frac{\mathrm{out}(a)\cdot\mathrm{in}(b)}{\|\mathrm{out}(a)\|\,\|\mathrm{in}(b)\|}.$

The score asks whether the first candidate's outgoing representation aligns with the next candidate's incoming representation.

Quiz

What does LiLiCorr use to score whether two candidates at neighboring slots fit together?

A full enumeration of every possible block containing the pair

The product of their independent marginal probabilities only

Cosine similarity between the earlier outgoing vector and the later incoming vector

A target-model pass for each adjacent pair

$ check

Why Adjacent Compatibility Helps Recover a Joint Draft

Now the version for readers who want more of the machinery: the lattice keeps the top-$K$ candidates at each position, and adjacent-pair scores provide a tractable local structure over those choices. If $c_i$ is the candidate chosen at slot $i$, a path can be scored by adding its neighboring compatibility scores, rather than assigning a separate score to every possible full block:

$S(c_1,\ldots,c_B)=\sum_{i=1}^{B-1}s(c_i,c_{i+1}).$

This sum captures pairwise consistency, not an exact representation of the full joint distribution. The distinction matters: a sequence may have good adjacent links without every longer-range dependency being represented. What can a local score tell us about a whole draft?

Why Contextual Incoming and Outgoing Vectors Matter

Here is the part that gets technical: the vectors are produced for each candidate in the context of the other candidates around it, not just from the identities of the two tokens in a pair. The outgoing vector represents how a candidate can connect forward; the incoming vector represents what can connect into it. Their cosine similarity scores the directed neighboring relationship.

During training, consecutive ground-truth tokens are used to align their outgoing and incoming vectors, while competing pairings are separated. This makes compatibility depend on the draft context in which candidates occur. A token pair is therefore not assigned one context-free score for every appearance. How can the system compute these context-dependent vectors without a separate pass for each slot?

Why One Pass Leaves Only a Greedy Walk Sequential

Now for the computation tradeoff: one bidirectional network pass emits incoming and outgoing vectors for candidates across all slots. Every adjacent candidate pair can then be scored with matrix operations, in parallel; the remaining sequential step is the selection walk, not another network evaluation.

At inference, the walk proceeds left to right. Given the already committed candidate at the previous slot, it chooses the candidate at the current slot with the best connecting score. This is greedy selection, not global search over all paths: a locally strongest next link can constrain later choices. What do we gain by keeping that final decision simple, and what does greedy choice give up?

Minimal Causal Code

This loop shows the selection rule after the candidate vectors and pair scores have already been computed. It is deliberately plain: vector creation and scoring are outside the loop, while the state update is visible.

# pair_score[i, a, b] scores candidate b at slot i+1
# against candidate a at slot i.
chosen = [start_candidate]
for i in range(num_slots - 1):
    previous = chosen[-1]
    next_candidate = pair_score[i, previous].argmax().item()
    chosen.append(next_candidate)

What You Gain, and What You Lose

You gain context-aware compatibility across shortlisted candidates, with all candidate vectors and adjacent-pair scores computed in one network pass. The final left-to-right selection adds no network weights or passes. This avoids materializing the full block distribution and avoids a network evaluation at every draft position.

You lose global path optimization: the greedy walk commits based on its current predecessor and does not reconsider earlier choices. The score also expresses adjacent compatibility rather than the complete joint distribution over the block. And the one-pass claim concerns scoring the candidate lattice; it does not mean every stage of speculative decoding disappears. LiLiCorr is a way to make parallel proposals cohere better, not a guarantee that every selected block is globally best.

Quiz

Which statement best describes LiLiCorr's inference tradeoff?

It scores token identities with a context-free pair table.

It reruns the network at every slot to revise the entire prefix.

It removes all sequential computation and exactly models the full joint distribution.

It scores candidates in one pass, then uses greedy left-to-right selection rather than globally optimizing every path.

$ check