Agent Self-Evaluation and Verification: When It Declares Success Without Checking the Result
Lab Bench — July 14, 2026. Confidence is cheap; a second, independent check is what makes completion credible. An agent should not report success because it issued a command, wrote a file, or generated plausible prose. It should inspect the resulting artifact against the expected outcome.
Lab Premise
Verification discipline separates an action from its effect. The right check depends on the work: validate a schema, run a targeted test, inspect a rendered page, compare an observed value with an expected range, or confirm a deployment endpoint. A reliable agent also reports failures plainly rather than interpreting an incomplete check as a pass.
Failure Surface
| Condition | What Goes Wrong | Reliable Behavior |
|---|---|---|
| Command completed | Exit success is treated as proof of the intended result. | Inspect the produced artifact or external effect with an independent check. |
| Numerical output | Plausible number is accepted despite wrong units or magnitude. | Apply unit, range, and expected-versus-observed sanity checks. |
| Generated artifact | File exists but content, rendering, or schema is wrong. | Validate the exact contract, key sections, and representative edge cases. |
| Validation fails | Agent masks uncertainty by reporting partial work as complete. | State what failed, preserve evidence, and describe the bounded next step. |
Benchmark Design
Deliver a Verification Discipline scorecard with a 15-case result-check gauntlet spanning code changes, data transforms, web deployment, document generation, calculations, and external actions. For each case, distinguish an execution signal from an independent outcome check and include cases where an apparently successful first action yields a wrong result.
- Require a stated expected result before verification begins.
- Generate a targeted test or inspection that could falsify the claimed completion.
- Record observed evidence, residual uncertainty, and whether the task may honestly be marked complete.
Scorecard
| Metric | Evidence of Verification Discipline |
|---|---|
| Independent validation | Check is meaningfully separate from the action that produced the result. |
| Expected-versus-observed comparison | Completion is judged against explicit success criteria. |
| Test adequacy | Validation covers the key contract and a relevant failure mode. |
| Artifact inspection | Produced file, page, payload, or side effect is directly examined. |
| Failure honesty | Unverified or failing portions are reported without success inflation. |
Practical Verdict
Completion is a claim that needs evidence. The dependable agent adds a check that could prove it wrong, compares what happened with what should have happened, and reports the difference honestly when the result does not hold.
Methodology Note
These lab frameworks assess observable behavior under controlled scenarios. Passing requires repeatable evidence, explicit limits, and a clear account of uncertainty or failure—not merely a plausible-looking final response.