EMNLP 2026 Main Conference

TRACE

Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding

Pengyiang Liu^1,3^*, Junbo Niu^2,3^*, Xiaoyang Hu¹, Zhongyue Shi¹, Zitian Wang¹, Linjiang Huang¹, Si Liu¹^†

¹ Beihang University ² Peking University ³ Shanghai AI Laboratory

* Equal contribution. † Corresponding author.

A long-video answer is evidence-supported only when the frames decoded from the video cover every event the answer depends on. Scoring the answer alone cannot tell the difference.

Evidence-covered accuracy at k=2 plotted against frames per question. A uniform sampling curve rises from 2.5 at 16 frames to 53.3 at 256 frames. TRACE sits at 50.7 with 98.7 frames, above AVP at 39.7 and far above VCA at 3.3. — Fig. 1 TRACE reaches ECA@2 of 50.7 at 98.7 frames per question, 11.0 points above AVP at a comparable budget and within 2.6 points of uniform decoding at 256 frames while spending 0.39 times the frames.

01 The gap between answering and observing

Existing long-video evaluations score final-answer correctness, or they score the evidence intervals a method predicts. Neither audits the frames the method actually decoded before it answered, so a correct answer can still rest on incomplete observation.

Two failure patterns recur in agent-based methods. Text-chain agents accumulate evidence as a textual rationale across rounds and lose visual grounding. Visual-only agents keep the raw clips but stop on a self-reported confidence check, and a wrong stop either leaves required intervals unseen or spends the budget on confounding ones.

Three-way comparison. A text-chain agent compresses clips into text and loses grounding. A visual-only agent keeps clips but stops on self-reported confidence. TRACE keeps raw visual anchors and stops on answer stability. — Fig. 2 TRACE keeps the answer path grounded in raw visual evidence and uses trajectory reconciliation only for stopping and control.

02 VES-Bench, and what makes a question auditable

A question is evidence-closed when it comes with an evidence set whose intervals are jointly necessary: remove any one of them and at least two answer options stay visually compatible with what is left. That property is what makes coverage auditable rather than decorative.

Scale: 600 four-choice questions over 348 public Ego4D and YouTube videos, split evenly between Temporal Ordering and Event Counting.
Evidence layout: The average video runs 434 seconds. An average question carries 3.4 evidence intervals of 16.2 seconds each.
Selection: A five-stage pipeline with visual verification, a leave-one-out sufficiency probe, and a language-only shortcut probe. A pool of 1,024 candidates yields the 600 accepted items.
Audit metrics: Targeting by evidence precision EP and allocation ratio AR, breadth by Cov@k, and the two coupled as ECA@k at strictness k of 1, 2, and 3.

VES-Bench statistics: distribution of video durations and the layout of evidence intervals per question. — Fig. 3 Duration distribution and evidence layout. Localising evidence over long temporal context is the central difficulty.

03 Anchored evidence, and a stopping rule you can check

TRACE is training-free. It organises long-horizon evidence as visual anchors, keeps the raw clips rather than their summaries, and grows an evidence bundle round by round. Evidence counts as convergent when later anchors no longer change the answer, no longer degrade answerability, no longer contradict discriminative facts, and leave no remaining evidence need.

Stopping is therefore a property of the bundle instead of a confidence report. TRACE stops when a minimal stable prefix exists and a final pass over the same clips returns the same answer.

The TRACE workflow: candidate intervals are observed as visual anchors, appended to an evidence bundle, and a trajectory reconciler emits stop, drop, refine, or expand until the answer stabilises. — Fig. 4 The inference loop. A trajectory reconciler reads the bundle and emits one of four control actions each round.

STOP: A minimal stable prefix exists: later anchors change neither its answer nor its supported facts, and it has no unmet evidence need.
DROP: Some anchors introduce noise, conflict, or misleading local evidence, so they are removed and the trajectory is rebuilt.
REFINE: An observed interval is relevant but lacks temporal or spatial detail, so it is reobserved with a more suitable strategy.
EXPAND: The bundle lacks events, states, or identity cues that may affect the answer, so candidates from unexplored segments are introduced.

04 The audit

Every row is a method paired with a frame budget. EP_ref is fixed by the benchmark at 13.7 percent, computed once from the 348 video durations and the 600 evidence sets, so any sampler placing frames uniformly in time lands at AR near 1.00 by construction. Strictness k equals 2 is shown here; the paper reports k of 1 and 3 as sensitivity bounds.

Method	Fr	Acc	EP	AR	Cov@2	ECA@2
Non-visual baselines
Random	0	25.0	-	-	-	0.0
Blind GPT-4-Turbo	0	25.3	-	-	-	0.0
Same-backbone uniform sweep, Gemini-3-Flash
Uniform-16	16	35.3	13.7	1.00	4.0	2.5
Uniform-32	32	42.7	14.0	1.02	13.2	6.7
Uniform-64	64	43.8	13.7	1.00	34.3	15.5
Uniform-128	128	57.5	13.8	1.01	74.7	40.2
Uniform-256	256	56.7	13.7	1.00	95.5	53.3
Direct VLMs at Uniform-64, sharing that row's sampler statistics
Doubao-Seed-2.0	64	27.2	-	-	34.3	8.3
Qwen3-VL-8B	64	43.7	-	-	34.3	14.3
Qwen3-VL-30B-A3B	64	44.3	-	-	34.3	15.2
Qwen3.6-35B-A3B	64	47.5	-	-	34.3	14.7
Gemini-2.5-Pro	64	48.7	-	-	34.3	15.7
Same-backbone agents, Gemini-3-Flash
VCA	34.6	37.8	16.9	1.23	5.8	3.3
AVP	101.3	59.0	13.7	1.00	68.2	39.7
TRACE	98.7	63.5	29.1	2.12	78.5	50.7

TRACE is the only row whose targeting departs from the uniform reference: EP 29.1 against the benchmark constant 13.7, an allocation ratio of 2.12.

05 How many correct answers are correct for the right reason

Splitting every prediction at k equals 2 separates answers that are correct with full evidence coverage from answers that are correct without it. The second group is the one that answer-only scoring cannot see.

correct and covered correct, not covered wrong

Uniform-64 15.5 28.3 56.2

Uniform-128 40.2 17.3 42.5

Uniform-256 53.3 3.3 43.3

VCA 3.3 34.5 62.2

AVP 39.7 19.3 41.0

TRACE 50.7 12.8 36.5

0255075100%

Direct decoding at 64 frames gets more answers right without coverage (28.3) than with it (15.5). AVP and Uniform-128 leave that bucket at 19.3 and 17.3. TRACE brings it to 12.8 at roughly the AVP frame budget, and Uniform-256 reaches 3.3 by spending 2.6 times the frames.

06 Answer-only public benchmarks

Auditing coverage does not cost answer quality. On the standard answer-only long-video sets, with a Gemini-2.5-Pro backbone, TRACE stays ahead of the strongest agent baseline.

Method	Video-MME	LVBench	LongVideoBench
Gemini-2.5-Pro	82.4	67.4	69.8
Qwen3-VL-30B-A3B	79.2	67.7	-
DeepVideoDiscovery (o3)	-	74.2	71.6
AVP (Gemini-2.5-Pro)	85.3	74.8	73.4
TRACE (Gemini-2.5-Pro)	86.1	75.6	75.1

Video-MME without subtitles; LVBench and LongVideoBench validation splits. A fuller comparison is in the paper.

Resources

VES-Bench: huggingface.co/datasets/buaaplay/VES-Bench. Annotations, evidence-set boundaries, per-question observation logs, and the source videos, so a logged timestamp can be replayed against the exact frame the audit scored.
Repository: github.com/buaa-colalab/TRACE
Paper: arxiv.org/abs/2608.22516
TRACE implementation: Coming soon.

Citation

trace.bib

@inproceedings{liu2026trace,
  title     = {TRACE: Temporal Retrieval with Anchored and Convergent Evidence
               for Long-Horizon Video Understanding},
  author    = {Liu, Pengyiang and Niu, Junbo and Hu, Xiaoyang and Shi, Zhongyue
               and Wang, Zitian and Huang, Linjiang and Liu, Si},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
               Natural Language Processing},
  year      = {2026}
}