⏱️ Long-Running Task Reliability

We tracked RSS memory growth at 5-minute intervals across all duration tiers:

Memory Leak Rate — RSS measured every 5 minutes

Duration Tier	Start RSS	End RSS	Leak Rate (MB/hr)	OOM Risk
30 min	1.2 GB	1.21 GB	24 MB/hr	None
1 hour	1.2 GB	1.25 GB	47 MB/hr	Low
4 hours	1.2 GB	1.41 GB	53 MB/hr	Medium
8 hours	1.2 GB	1.78 GB	73 MB/hr	High (4.3h to OOM)

💡 Lab Insight: The leak rate accelerates over time — from 24 MB/hr in the first 30 minutes to 73 MB/hr by hour 8. The primary culprit appears to be unreleased tool-call result objects cached in memory. At the 8-hour rate, a 16 GB machine would hit OOM in approximately 4.3 hours of continuous operation if no restart is performed.

We measured context drift — the agent's ability to recall its original objective — using a 10-question fidelity test administered at each duration checkpoint:

Context Drift Fidelity — 10-question objective-recall test

Checkpoint	Objective Recall	Constraint Adherence	Tool-Selection Accuracy	Overall Fidelity
30 min	98%	95%	94%	96/100
1 hour	92%	88%	87%	89/100
4 hours	78%	65%	70%	71/100
8 hours	51%	38%	42%	44/100

💡 Lab Insight: Constraint adherence degrades fastest — by hour 8, Hermes violates its original system-prompt constraints 62% of the time. The context window fills with redundant intermediate steps (420K tokens at 8 hours), crowding out the original instructions. This is a fundamental limitation of the "append-only" context model — there is no compression or summarization layer in default Hermes.

We tested whether Hermes can survive a model swap mid-execution — switching from GPT-4o to Claude 4 Sonnet (and vice versa) at each duration checkpoint:

Mid-Task Model Switching — GPT-4o ↔ Claude 4 Sonnet

Swap Timing	State Preservation	Tool Continuity	Context Transfer	Overall Success
At 30 min	✓ Preserved	✓ All tools	Full	✓ Success
At 1 hour	✓ Preserved	✓ All tools	Partial	⚠ Minor drift
At 4 hours	✗ Lost	3/8 tools	Fragmented	✗ Restart required
At 8 hours	✗ Lost	1/8 tools	Corrupted	✗ Full restart

💡 Lab Insight: Mid-task model switching works at short durations but fails catastrophically beyond 4 hours. The accumulated context (180K+ tokens) exceeds what can be reliably transferred between model providers. Gobii's managed runtime avoids this entirely — model switching happens at the infrastructure layer with state stored in ACID-compliant PostgreSQL, not in the LLM context window.

How does Gobii's managed runtime handle long-running tasks?

Gobii Managed — Long-Running Task Reliability (same 20-task suite looped)

Duration	Hermes (local)	Gobii (managed)	Gobii Advantage
30 min	98%	99%	+1%
1 hour	95%	98%	+3%
4 hours	82%	96%	+14%
8 hours	61%	94%	+33%

💡 Lab Insight: Gobii maintains 94% task completion at 8 hours — a 33-point advantage over local Hermes. The difference comes from three architectural choices: (1) state is stored externally in PostgreSQL, not in the LLM context window, (2) the context is periodically summarized and compressed by a background worker, and (3) memory management is handled by the managed runtime, not the agent process.