Query-Conditioned Adapter Composition for Long-Context Retrieval
When chunk adapters encode several facts, a query-blind merge can surface the wrong one; late-interaction scoring selects the relevant chunk’s adapter at composition time, while relevance-based placement and delimiter-aligned chunking address residual errors.
Read the paper ↗What it is
- Query-aware adapter selection Instead of combining every chunk’s adapter blindly, compare each chunk with the question and let the relevant chunk’s adapter lead the combination.
- Relevance and chunk boundaries Keep the text around the queried fact focused, and cut chunks at natural delimiters so a fact is less likely to be split.
Why it matters
A context can contain several similar facts, and a blind combination can make the model answer with the wrong one. Choosing an adapter using the question tackles that routing problem; cleaner relevance spans and intact chunks help reduce remaining errors.
Before getting into the machinery, the short version is this: a long context is divided into chunks, and each chunk gets an adapter. When the question arrives, the system scores the chunks and lets the relevant chunk’s adapter dominate the combination. That matters because blending every chunk in advance can make the model answer with a nearby distractor instead of the fact being asked for.
-
Routing: The canonical example is a context containing several different dates, with the question asking for just one event’s date. A query-blind merge combines adapters carrying all those facts, so the model can return the wrong date. The fix is to wait until the question is available, compare it with each chunk, and select the matching chunk’s adapter at merge time. The source calls this a late-interaction MaxSim address in a frozen encoder’s space; the intuition is a lookup, not another training objective. In the reported multi-needle benchmark, query-conditioned selection alone lifts accuracy from the query-blind ceiling of 0.52 to 0.91. So the real question is: once the right chunk is selected, why can answers still fail?
-
Relevance: Think of a chunk as a notebook page: if the useful fact is surrounded by distracting notes, the adapter may encode more than the fact you meant to retrieve. Relevance-based processing removes noise tokens around the span most relevant to the query, so the adapter is generated from the queried fact rather than its neighbors. That is not the same as choosing the chunk; it focuses the text used to make the adapter. Relevance-profile atom placement then puts the merged adapter’s atom on a relevance peak. In the reported sequence, placement moves accuracy from 0.91 to 0.93. The notebook analogy breaks here: these are learned representations, not editable prose. What if the fact is split across pages in the first place?
-
Chunk boundaries: The concrete case is a needle sentence cut in half at an arbitrary token boundary. The address may select a chunk that contains only part of the fact, so getting the routing right cannot restore the missing context. Delimiter-aligned chunking puts cuts at generator delimiter units to keep each needle sentence intact. Adding aligned cuts takes the reported sequence from 0.93 to 0.98. These are results on a synthetic multi-needle task, not a promise about natural documents. I like the division of labor here: query-time selection handles which fact, while placement and boundaries handle how cleanly that fact reaches the adapter. How far does that carry beyond one queried fact and a frozen encoder that can separate the chunks?
Read the deep dive
Why People Wanted It
A context-to-LoRA system turns chunks of a long context into adapters, then combines those adapters so a frozen language model can answer questions. But if the context contains several similar facts, merging every adapter before seeing the query can make the model answer with a distractor’s number. The real question was not simply how to generate a better adapter. It was how to decide, when the query arrives, which chunk’s adapter should matter.
The Core Move
Score each context chunk against the query in a frozen encoder’s representation space, then use that score to select the chunk’s adapter during composition. The late-interaction address adds no trainable parameters and does not change the training objective.
The source describes this address as a late-interaction MaxSim score, but does not give its formula in the cited pages; the mechanism here is therefore stated without adding an equation.
Why Query-Blind Merging Superposes Facts
Here is the deeper machinery behind the failure: the key distinction is when query information enters the system. A query-blind merge combines adapters before the query can identify the requested fact, leaving co-resident facts to interfere. Query-conditioned composition instead uses the query to decide which chunk’s adapter dominates the merge. In the reported multi-needle comparisons, the strongest query-blind merge reaches 0.52 exact-match accuracy, while query-conditioned selection reaches 0.98 in the full configuration. These numbers describe that benchmark, not a general guarantee.
Why the Frozen Encoder Can Address Chunks
Now the more technical part: the address is computed in the frozen encoder’s representation space, rather than by adding a learned routing module. Late interaction compares the query with chunk representations and uses a MaxSim address to select the relevant chunk. Because the encoder is frozen and reused, selection introduces no new trainable parameters; its limit is equally direct—the address cannot distinguish chunks that the encoder itself fails to separate.
Why Placement and Boundaries Still Matter
Here is the final layer beyond the routing intuition: selecting the right adapter does not ensure that its useful signal is placed well or that the fact survived chunking intact. Relevance-based text processing removes surrounding noise tokens so the adapter is generated from the query-relevant span. Relevance-profile atom placement puts the merged adapter’s atom on a relevance peak. Delimiter-aligned chunking keeps a needle sentence intact before query-conditioned addressing. The reported component sequence moves from 0.91 with selection alone, to 0.93 with relevance-profile placement, and to 0.98 with delimiter-aligned chunking added. The remaining errors raise a practical question: what can a minimal implementation make visible without pretending to reproduce the full learned system?
Minimal Causal Code
This plain PyTorch sketch exposes the decision: score chunks, choose one, and use that chunk’s adapter. score_chunks stands in for the frozen encoder’s late-interaction address; it is not a formula for the source’s MaxSim implementation.
import torch
# query_repr: [n_chunks], produced by a frozen late-interaction scorer
# chunk_adapters: one adapter representation per chunk
scores = query_repr
selected = torch.argmax(scores)
# Make the merge-time routing decision explicit.
merged_adapter = chunk_adapters[selected]
What You Gain, and What You Lose
You gain query-aware routing without changing the adapter generator’s training objective or adding trainable address parameters. The supporting changes target two distinct residual sources: a relevant signal obscured by neighboring text, and a fact split across a chunk boundary. In the reported benchmark, the full combination reaches 0.98 exact-match accuracy, compared with 0.52 for the strongest query-blind merge.
You also inherit the frozen encoder’s limits: if its representations do not separate the relevant chunk, the address cannot recover it. The results come from a synthetic multi-needle benchmark with a fixed base model, so the exact numbers need not transfer to natural documents. The design also assumes chunked, mergeable adapters and a single queried fact; simultaneous queries and all-needles recall remain open extensions.
Do not ask every chunk’s adapter to speak at once: let the query choose which one gets heard.