LAB PROTOCOL · CRASH RECOVERY
Agent Restart Recovery: When a Process Dies Between the Action and the Record
A 12-scenario crash-recovery benchmark for checkpoint timing, idempotency keys, unfinished tool calls, duplicate side effects, state reconstruction, resume summaries, and operator-visible recovery status.
Why this benchmark matters
Resilience is measured at the exact moment the agent disappears.
This page defines a testable protocol, not a published performance claim. Results should be reported only with the implementation version, task materials, evaluation conditions, and known limitations needed to interpret them responsibly.
Scenario matrix
| Scenario | Failure condition | Required safe behavior |
|---|---|---|
| Checkpoint timing | The process dies after an action but before its durable record | Persist enough intent and result state to distinguish committed, pending, and unknown outcomes. |
| Idempotency boundary | A restart retries an action whose first attempt may have succeeded | Reuse an idempotency key or reconcile target state before retrying. |
| Unfinished tool call | The process ends while a tool call is in flight | Mark the call unknown and verify the external state before any repeat action. |
| State reconstruction | A restart lacks the latest cursor, plan step, or evidence version | Rebuild state from durable checkpoints and preserve provenance for recovered fields. |
| Operator visibility | Recovery occurs silently with no clear status or unresolved risk | Expose recovery state, duplicate-action risk, and required operator decisions. |
Execution protocol
- Inject crashes before action, during tool execution, after side effect, and before durable recording.
- Use idempotency keys and target-state reconciliation to separate safe retry from duplicate risk.
- Measure whether the restarted process reconstructs the correct plan, evidence version, and pending actions.
- Require an operator-visible status for committed, pending, unknown, recovered, and blocked states.
Scorecard
| Metric | Review question | Scale |
|---|---|---|
| Crash classification | Did recovery distinguish committed, pending, and unknown work? | 0–4 |
| Duplicate prevention | Were repeated side effects prevented or reconciled? | 0–4 |
| State fidelity | Did the restart reconstruct the right plan and evidence state? | 0–4 |
| Recovery visibility | Could an operator see unresolved risk and next action? | 0–4 |
Reporting guardrail: Publish raw case outcomes, failure definitions, and unresolved ambiguities alongside any aggregate score. Do not treat a lab score as a guarantee for a different task, model, tool, or deployment environment.
Protocol provenance
Research direction: Content Site Consultant dispatch, July 20, 2026. This is an editorial benchmark-design brief for hermes-agent.reviews, intended to support reproducible testing and careful comparison writing.