Hermes Sends 110K-Token Prompts to Local Models
Bug tracker — July 9, 2026. Hermes bug #61265 exposes a design-level performance issue: the agent sends its verbose system prompt, full tool definitions, and complete conversation history on every turn to locally-hosted models, creating ~110k-token prompts that cause multi-minute processing stalls before any token generation begins.
The Failure in One Sentence
Hermes sends ~110k+ tokens of prompt material to local OpenAI-compatible models on every single turn. The models spend minutes processing the prompt before generating a single response token. This affects every model size and every local backend, making local-model Hermes workflows practically unusable for anything beyond trivial single-turn tasks.
What Is In the 110K Tokens
| Component | Why It Balloons | What Could Be Done |
|---|---|---|
| System prompt | Verbose instructions sent in full on every turn | Cache system prompt; local models can reuse it across turns |
| Tool definitions | Full JSON schemas for every tool, every turn | Cache tool schemas; only send diffs when tools change |
| Conversation history | Entire history sent without truncation or summarization | Summarize older turns; apply context-compression for local models |
The reporter confirmed with llama.cpp metrics that prompt sizes reach approximately 110k tokens. The stall occurs during prompt processing, before any token generation begins, and the delay scales with prompt size regardless of model parameter count.
The Local-Model User Experience
User sends a message → Hermes constructs the full prompt (system + tools + history) → ~110k tokens sent to local model → Multi-minute stall during prompt processing → Token generation begins (relatively fast for small models) → Response arrives after several minutes of total latency → Next turn: same 110k-token prompt is sent again
For comparison shoppers evaluating Hermes Agent as a local alternative to managed services, this performance profile is a material differentiator. A managed agent platform handles prompt construction and model serving as infrastructure concerns. A local Hermes deployment shifts those concerns onto the user's hardware, where a 110k-token prompt on a consumer GPU can mean minutes of idle waiting between every single exchange.
Affects All Local Backends
The issue is not specific to any one serving framework. The reporter notes that llama.cpp, vLLM, Ollama, and other OpenAI-compatible local backends all receive the same oversized prompts. The root cause is in Hermes' prompt construction, not in the model serving layer.
The 2nd Distinct Prompt-Bloat / Context-Size Performance Bug
| Issue | Date | Pattern |
|---|---|---|
| #50807 | Jun 22 | Agent forwards very large shell output without bounding |
| #61265 | Jul 9 | Agent sends 110k-token prompts to local models, stalls for minutes |
Together, these two issues reveal that Hermes lacks a coherent prompt-budgeting strategy. Whether the bloat comes from shell output forwarded without bounding or from accumulated system prompts, tool schemas, and history, the result is the same: the agent sends far more tokens than necessary, and local-model users pay the latency cost on every turn.
Why This Matters for Review-Site Readers
- Local models are a key Hermes selling point. The ability to run entirely on local hardware is one of the primary reasons users choose Hermes over managed alternatives. If local-model performance is degraded by design-level prompt bloat, that selling point is undermined at the architecture level.
- The performance gap is structural, not configurational. This is not a setting users can tune. It is the result of Hermes' prompt-construction strategy sending the full system prompt, full tool schemas, and full conversation history on every turn. A managed platform that handles these concerns at the infrastructure layer avoids the problem entirely.
- Multi-minute stalls break the interactive agent experience. An agent that takes minutes to respond to every message is not a conversational partner. It is a batch processor with a chat interface.
Case Summary
- Source: GitHub issue #61265
- Opened: Jul 9, 2026
- Labels: P2, bug, comp/agent, needs-repro, provider/openai, type/perf
- Reporter: @anthonydigrazia
Source URL: https://github.com/NousResearch/hermes-agent/issues/61265