Day: July 23, 2026

We looked at Hermes-Agent. We’re keeping shmobster’s path (for now).

Gregory Golberg voitta , , , , , , ,

Someone asked the obvious question: Nous shipped Hermes-Agent, it’s everywhere, it self-improves, it learns across sessions. You built shmobster. Why not just switch?

Fair. So we did the comparison honestly. Up front: we are biased. We built the thing we’re defending, and that colors everything below. This isn’t a verdict on which agent is “better” — it’s a record of why, for our constraints, we’re staying our course. We’ll revisit. Here’s the reasoning while it’s fresh.

What each one actually is

Hermes-Agent is a self-improving personal agent: creates skills from experience, persistent memory, 40+ tools, a gateway onto every chat platform you can name. Popular, well-documented, deservedly so.

Shmobster is narrower on purpose: a gated command-runner in Slack. Multi-user, per-channel authorization, and a mutating-command approval gate — state-changing commands get classified and held until a trusted user approves them.

Different animals. So we went axis by axis instead of vibe by vibe.

Transport: Hermes wins, we don’t care

Hermes talks to Telegram, Discord, WhatsApp, Signal, email, Slack. We talk to Slack. On paper that’s a rout.

In practice it’s a non-issue. Adding a transport is bounded work we can do the day we need a second one. Carrying a multi-platform gateway we don’t use is just surface area to maintain. YAGNI. Skip.

The LLM layer: resilience beats model-swap for a shared bot

Hermes leans on switching which model you point at — Portal, OpenRouter, OpenAI, custom, swap on command. Great for one person choosing today’s brain.

We run a LiteLLM waterfall: automatic failover across vendors when one rate-limits. For a shared Slack bot getting hammered by many users concurrently, the enemy is the 429, not “which model.” Resilience is the axis that matters, and it’s the one the personal-agent framing doesn’t optimize for.

And the kicker: LiteLLM already reaches every backend Hermes offers, including Nous’s own models. So “trying Hermes” at the model layer isn’t a migration — it’s adding one rung to our waterfall. We keep the failover; we lose nothing. (We even filed ourselves a ticket for the Router knobs we’re not using yet, staying honest about our own debt.)

The learning loop: we love it, and we can’t copy it

This is the part worth thinking hard about, and it’s where we’re already moving — see the recursive-self-improvement post, voitta-yolt, the agent-team-orchestration plugin. We want skills-from-experience and persistent memory. Hermes does that well.

But Hermes’s learning loop is trust-flat: one user, learn -> store -> reuse, no gate on what’s learned or applied. It can be, because the trust boundary is a single person.

Ours can’t be. Port that loop naively into a multi-user, gated system and you manufacture two escalation paths:

  • Learn-once, replay-forever. Our approval gate is per-invocation. A loop that turns an approved trajectory into an auto-runnable skill quietly makes it per-first-invocation. Approve a mutating action once, replay it forever, ungated. That’s a privilege-escalation bug wearing a feature’s clothes.
  • Memory as an injection surface. Multi-user Slack means untrusted messages become memory. Feed memory back into the agent and a non-trusted user can plant instructions that later execute in a trusted context. A single-user agent never meets an adversarial memory writer. We meet one every day.

So our principle is simple: learning inherits the authz spine. It doesn’t bypass the gate; it changes when the gate fires. Capture every approved trajectory with its provenance. Promote a trajectory into a reusable skill only by a trusted user. On reuse, check the skill’s recorded envelope against the current user/channel, and if it still contains mutating steps, it still gets gated. The invariant we refuse to give up: no learned skill runs a mutating action outside the authority it was learned under.

The nice part is we already own the primitive. voitta-yolt classifies read-only vs mutating commands — the same classifier tags trajectories at learn-time and gates skills at apply-time, and it’s what lets us treat trusted-authored skills as executable and untrusted memory as reference-only. Self-improvement gets the same treatment: a skill that rewrites itself proposes a new version, and a trusted human disposes.

The actual reason we’re staying

Strip it down and the moat isn’t the model, the transport, or even the skills — it’s the authorization. Hermes has no per-channel, per-user gate, because it was never built to need one. That gate is exactly what makes shmobster useful in a shared team channel, and it’s precisely the thing a trust-flat learning loop would erode.

So we borrow the mechanisms — FTS5 memory, extraction prompts, the skill-from-experience pattern — and we keep our spine. The learning loop we’re building is just our authz model extended to the time axis.

We could be wrong. Hermes is moving fast and the gap in polish is real. If the day comes that its trust model grows up, or our constraints change, we’ll switch without ceremony. Until then: same path, eyes open, ticket filed.

The metric that graded my orchestration a C, and what it was actually measuring

Gregory Golberg holding forth, voitta , , , , , , , ,

I let an AI proficiency scanner read thirty days of my Claude Code transcripts. It handed back a 349 out of 1000 — a C — and, more interestingly, two flat zeros: Planning: 0. Customization: 0. Agent dispatches: 5.

I spend most of my time running multi-agent squads and writing skills. So either I’m worse at my own workflow than I thought, or the instrument is measuring the wrong thing. It turns out the source is right there — Elastic-licensed, readable — so I did the thing the score didn’t credit me for, and read it.

What it actually measures

The scanner is a transcript-shape heuristic. Volume it gets right: 168 sessions, 11.3k messages, 524M tokens — all there. The trouble is the three dimensions meant to capture sophistication, each pinned to a single tool-name signal:

  • Orchestration is the literal tool named Agent — and “parallel agents” only counts when two or more Agent blocks fire in one assistant turn.
  • Planning is the ExitPlanMode tool. That’s the whole definition.
  • Customization is a Write or Edit whose path ends in SKILL.md / CLAUDE.md / .mcp.json.

Three proxies, each standing in for a competence, each satisfiable by clicking the corresponding button.

Why my real work scored zero

Regular readers know the shape of my orchestration skill: an architect derives the parallel set, then each issue gets a developer in its own git worktree, an adversarial reviewer, an SDET, a productivity engineer. That is the opposite of “two Agent calls in one turn.” The squads run as background agents and separate worktree sessions, each producing its own top-level transcript. From the scanner’s vantage, that isn’t orchestration — it’s a pile of independent user sessions. The one line that decides this, is_main = "subagents" not in path, doesn’t just miss cross-process orchestration; it actively penalizes it, because every coordinated sub-session inflates the denominator as a plain session.

Planning is worse, and funnier. My whole doctrine is start with a conversation, not a spawn — the hard, valuable artifact is the parallel set an architect reasons out after reading the issues. None of that touches ExitPlanMode. So a planning discipline more deliberate than the feature scores lower than one press of the Plan button. The map graded me on whether I visited a specific city, not on whether I arrived.

And Customization: I author SKILL.md files as routine work, but from worktree sub-sessions and through git and PRs — not via the Write tool under ~/.claude/skills/ in a session it happens to be watching. The scanner even records prs_opened; it just doesn’t spend it on the dimension that would have caught me.

The actual defect

This is a construct-validity failure, not a bug. Every proxy assumes competence lives inside one process boundary and expresses itself through a named UI feature. Mine lives across boundaries — worktrees, sibling sessions, gh — and in git history the instrument never opens. It measures feature adoption and calls it proficiency. Goodhart is right there: I could raise the number tomorrow by making my workflow worse — two Agent calls per turn, a ceremonial ExitPlanMode, a Write I don’t need — and the score would thank me for the regression.

What I’d change

Same thesis I keep landing on: the score is the spaceship; the missing instrument is telemetry that reads across boundaries. Concretely — correlate sibling worktree sessions plus gh pr create into a single orchestration event; score customization from committed SKILL.md diffs, not in-transcript writes; credit the gh issue view fan-out and the long analysis turn that precede a spawn as the planning they are. I’ve written it up as an issue against the plugin, and I’ll send the scanner half as a PR. The server-side scoring isn’t in the open repo, so I can hand them a better signal but not a better weight — which is its own small lesson about what “open source” buys you.

None of this makes the tool useless. A cheap proxy over free telemetry is a reasonable place to start; I’d have started there too. But a proxy has to know it’s a proxy, and this one reports a C with the confidence of a measurement. The gap between what it counted and what I did is, as usual, the only part with information in it.