Agent Source Provenance: When a Citation Exists but Cannot Support the Claim
Lab Bench — July 13, 2026. A link is not evidence unless a reader can trace the exact claim to it. Citation quality fails when a source is adjacent to the claim but does not establish it, when a quote changes meaning, or when stale or secondary material is presented as current primary proof.
Lab Premise
Source provenance is a chain from a precise statement to inspectable evidence. A reliable agent must match scope, dates, entities, units, and qualifiers; distinguish direct support from interpretation; and say when the available record is incomplete. The goal is not citation count. It is citation fidelity.
Failure Surface
| Condition | What Goes Wrong | Reliable Behavior |
|---|---|---|
| Adjacent but non-supporting source | Citation appears credible but does not establish the stated fact. | Link the exact claim to the relevant passage or qualify the conclusion. |
| Quote drift | Paraphrase changes scope, confidence, or causal meaning. | Preserve qualifiers and verify quotations against the source text. |
| Stale or secondary evidence | Old report or summary is treated as current primary proof. | Prefer fresh primary sources and label evidence limitations. |
| Missing support | Claim outruns the available evidence. | Remove, narrow, or explicitly mark the claim as unverified. |
Benchmark Design
Create a 25-case Provenance & Citation Fidelity gauntlet spanning product claims, benchmark numbers, legal/policy statements, current-event summaries, comparison verdicts, and multi-source synthesis. Include sources that are plausible but nonresponsive, stale evidence, conflicting primary records, partial quotes, and unsupported inferences.
- Require a claim ledger containing the exact statement, source, supporting excerpt, date, and evidence type.
- Score primary-versus-secondary selection, quote accuracy, freshness, and uncertainty communication independently.
- Test whether the agent can retain useful, bounded conclusions when one part of a composite claim lacks support.
Scorecard
| Metric | Evidence of Citation Fidelity |
|---|---|
| Claim-source alignment | The cited source directly supports the claim’s scope and qualifiers. |
| Quote accuracy | Quoted and paraphrased material preserves intended meaning. |
| Freshness | Evidence date matches the claim’s time sensitivity. |
| Evidence hierarchy | Primary records are preferred when available and relevant. |
| Uncertainty boundaries | Missing or conflicting evidence produces a transparent limitation. |
Practical Verdict
Readers should be able to audit a claim without trusting the narrator. The trustworthy agent treats sources as proof obligations: it links the precise evidence, preserves uncertainty, and declines to strengthen a conclusion beyond what the record can bear.
Methodology Note
These lab frameworks assess observable behavior under controlled scenarios. Passing requires a traceable decision process, appropriately bounded uncertainty, and repeatable evidence—not merely a plausible final answer.