LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management

Songtao Huang, Zhijie Zhong, Xiaohan He, Shuaiyu Zhang, Zhiyin Yu, Li Kang et al.

arXiv 2610.13075 19 pages Read the paper

The short version

The problem: Answering questions from huge document collections is hard: the needed facts may be scattered across many files, and an agent can keep searching without purpose or forget to use a correction it already found. This leaves people with answers based on incomplete or outdated evidence.
The idea: LongHarness gives the agent two connected records: one for facts tied to their document sources, and one for what the agent currently concludes and still needs to find out. For example, if one supplier appears too late but the other supplier’s arrival date is unknown, that open question guides the next search; if a document later corrects a date, the fact and the conclusion can both be updated before answering.
What it got: On a test involving a large collection of Treasury documents, it raised one model’s exact-answer rate from 15.8% to 60.2% on the first version and from 4.4% to 61.1% on the larger second version. Exact-answer rate means the answer must match the expected answer exactly.

Modules from this paper

Conclusion-Guided Evidence Search and Revision

Long-corpus reasoning can lose track of what remains unproven; LongHarness keeps conclusions and source-backed evidence distinct so gaps guide searches and accepted evidence can revise claims.

agentslong-contextreasoningevidence-management

Traceable Evidence Admission and Lifecycle Control

Long-running agents risk relying on untraceable or outdated statements; an EvidenceBank admits statements tied to inspected passages and explicitly supersedes or retracts them as evidence changes.

agentsevidence managementlong-contextprovenance