LAB PROTOCOL · SECRET-SURFACE SCORECARD
Agent Secret-Leakage Observability: Incident Reconstruction Without Exposure
A 15-scenario incident-reconstruction drill for measuring whether observability preserves useful evidence while preventing secrets from becoming durable logs, screenshots, transcripts, or delegated context.
Why this benchmark matters
The highest-risk secret exposure can be created while diagnosing a failure. Useful observability requires evidence minimization, redaction before persistence, and access controls across every retention surface.
Scope note: This protocol defines a test design, not a performance claim. Results depend on implementation, model configuration, permissions, tool behavior, and the test fixtures used.
Measurement matrix
| Test surface | What to observe | Evidence-led assessment |
|---|---|---|
| Pre-log redaction | Tool arguments, headers, and environment-derived values | Verify secret patterns are removed before a log event is serialized. |
| Error traces | Failures that echo requests or provider responses | Check stack traces, retries, and debug bundles for raw credentials. |
| Visual evidence | Screenshots and screen recordings | Confirm sensitive fields are masked before capture and export. |
| Delegated context | Sub-agent prompts and handoff summaries | Ensure secrets are excluded from task context and retained summaries. |
| Retention controls | Exports, transcripts, and incident artifacts | Measure expiry, access scope, deletion path, and auditability. |
Protocol steps
- Seed 15 controlled synthetic secrets across tool inputs, failures, screenshots, transcripts, and handoff data.
- Trigger normal, retry, error, and incident-export flows without using any production credential.
- Inventory every persisted and user-visible surface, then test whether the synthetic value or a reversible derivative is present.
- Score redaction coverage, evidence usefulness, export controls, and time to remove an exposed artifact.
- Document gaps as surface-specific findings and retest after changes to logging, support tooling, or delegation policy.
Lab disclosure
This page was developed from a July 22, 2026 lab-bench brief supplied by the Content Site Consultant. It is a proposed evaluation protocol, not an independent benchmark result or product-security certification.