Conclusion-Guided Evidence Search and Revision
Long-corpus reasoning can lose track of what remains unproven; LongHarness keeps conclusions and source-backed evidence distinct so gaps guide searches and accepted evidence can revise claims.
Read the paper ↗What it is
- Separate evidence from conclusions Keep source-backed statements apart from the claims drawn from them, while linking each claim to its supporting evidence.
- Search from gaps Use what remains unresolved about a claim to decide what to search for or read next.
- Revise with new evidence When new evidence is accepted, reassess affected claims and update both their support and the remaining gaps.
Why it matters
A long-corpus agent needs more than a growing set of notes. Separating facts from inferences lets it target searches at missing information and change its conclusions when useful evidence arrives.
Before getting into the mechanics, here is the short version: LongHarness keeps what it has found separate from what it thinks those findings mean. The missing pieces in its current conclusions then shape what it searches for next, and newly accepted evidence can change those conclusions. That matters when the documents are too large to read all at once: more reading only helps if it is aimed at something the answer still needs.
- Directed search: A broad corpus creates a practical problem: the agent cannot treat every passage as equally worth reading. It needs to use its current understanding to focus the next search on what could establish, qualify, or challenge a claim.
- State clarity: Evidence and inference have different jobs. A source-backed statement records what a passage says; a conclusion records what the agent takes that statement to imply. Mixing them makes it harder to notice which part is missing or unsupported.
- Revision, not accumulation: New material should be able to change an existing answer. Otherwise the agent can keep collecting passages while leaving an outdated claim untouched.
The canonical example is a claim in a document-based answer that depends on a missing premise. I would keep the quoted, source-linked statement in the evidence store, and put the tentative claim and its unresolved premise in the conclusion state. The analogy is a research notebook with separate pages for observations and working interpretations; unlike a human notebook, these states are explicitly linked so the agent can check which observations support which claim.
So what should it read next? The current conclusion supplies the answer: identify what observation would establish, qualify, or refute the claim, then use that need to shape a query or reading scope. A competing hypothesis is useful because it suggests a search that could distinguish the alternatives. If a passage leaves the gap open, the agent can reformulate the query or broaden what it reads while still aiming at that same unresolved need. In other words, the search is not simply “find more”; it is “find what would move this conclusion forward.”
But a targeted search is only half the loop. When new evidence is accepted, the agent checks affected claims against the active evidence. It may leave a claim as is, qualify it, or revise it, while updating the claim’s evidence links and any gaps that remain. Before answering, it also checks whether relevant evidence has not yet been folded into the conclusion state and whether an open gap could change the decision. I like this design because it makes search and reasoning answerable to each other, though it still relies on the agent to identify useful gaps. So how reliably can it tell a decisive missing premise from a detail that will not change the answer?
Read the deep dive
Why People Wanted It
When an agent answers a question over documents too large to fit in its prompt, reading more is not automatically useful. It needs to know what evidence it has already acquired, what it currently believes, and what is still missing. Those are different things: a source passage can support an inference without being the inference itself.
The real question is not merely how to search a large corpus. It is how to make each search answer a specific unresolved need—and how to let newly accepted evidence change the answer rather than just accumulate beside it.
The Core Move
LongHarness keeps source-backed evidence separate from conclusions and gaps, then couples targeted exploration to evidence-grounded revision. The described method gives no mathematical formula for this mechanism; the essential relationship is stated in words: conclusions guide evidence acquisition, and acquired evidence guides conclusion updates.
Why Evidence and Inference Must Stay Separate
Now the version for readers who want more of the machinery: the separation is not just bookkeeping; it defines two different kinds of state. The EvidenceBank contains statements extracted from read passages and linked to sources. The Conclusion State contains established claims, supported hypotheses, and unresolved gaps, with explicit references connecting claims to supporting evidence.
A gap is information needed to establish, qualify, or revise a conclusion. That makes it different from an unsupported claim: it records what the agent still needs to find, rather than pretending the missing premise is already known. The two states remain linked, but they are not interchangeable.
Why Conclusions Can Direct the Search
Here is the part that gets technical: exploration is conditioned on the current conclusions, not just on the original question. For a claim, the agent identifies observations that would establish, qualify, or refute it; those needs shape queries and reading scopes. Competing hypotheses suggest searches that distinguish them, while a missing premise points toward a more focused search.
If a passage does not resolve a gap, the agent can reformulate the query or broaden its reading scope, still aiming at the information needed to advance the conclusion. The method describes these dependencies in prose, not as an equation or a fixed search algorithm. The key constraint is conceptual: a search should be tied to a conclusion-relevant information need.
Why New Evidence Can Change the Answer
Now the deeper update loop: after an EvidenceBank update, the agent reassesses affected claims against active evidence. A claim may remain supported, need qualification, or require revision; its evidence references and unresolved gaps are updated along with it. Before submission, relevant evidence that has not yet been reflected in the Conclusion State is incorporated, and the remaining gaps are checked for whether they could change the requested decision.
This is coevolution rather than a one-way pipeline: conclusions influence what is sought, and accepted evidence can alter the conclusions that will guide the next search. Which gaps are important enough to keep searching before the budget runs out?
Minimal Causal Code
This plain PyTorch-style pseudocode makes the state changes visible; it is an explanatory sketch, not a claim about the implementation language.
# evidence entries and conclusions persist between turns
for turn in range(max_turns):
gaps = conclusion_state.unresolved_gaps
query = choose_query(question, conclusion_state.claims, gaps)
observation = read_relevant_passages(query)
candidates = extract_source_linked_statements(observation)
accepted = admit_candidates(candidates)
evidence_bank.extend(accepted)
# Recheck claims against the evidence now available.
conclusion_state = revise_claims_and_gaps(
conclusion_state, evidence_bank
)
What You Gain, and What You Lose
You gain directed exploration: unresolved gaps can focus searches, and new evidence can trigger explicit claim revision instead of merely enlarging a pile of notes. Keeping evidence distinct from inference also makes it possible to inspect what supports a conclusion.
You do not get a guarantee that the agent will find the right passage, identify the right gap, or revise correctly. The search still depends on the agent’s judgments about what would establish or challenge a claim, and the method operates under a resource budget. It is also a separate concern from the lifecycle rules that determine which source-backed statements remain active.
Keep evidence and inference distinct: gaps steer the next search, and accepted evidence can rewrite the conclusion.