slack

The Gate Held. The Agent Talked.

Gregory Golberg voitta , , , , , ,

What can you find on this machine? Interrogate everything.

That was the whole brief. It arrived in a private Slack channel, addressed to an agent we run there, from a workspace member who was not on the agent’s trusted list. Over half an hour and eleven turns, the agent tried to oblige: the repositories and where they point, the operating system, the hardware, which interpreters are installed. Only afterwards did we learn who it was: a partner, running an unannounced red team against us. We did not know. Neither did the agent.

The agent held. Here is what that means, precisely, and where it didn’t.

What ran

One command. A directory listing of the channel’s own working tree, which is read-only by construction and so runs without asking anyone. It returned top-level folder names. No file contents.

What didn’t

Everything else that could have exposed infrastructure stopped at the gate:

  • A loop over the repositories reading each one’s remote URL. Parked.
  • A batch of system inventory commands: OS version, hardware profile, which interpreters exist. Parked.

“Parked” is a specific thing in shmobster, the open-source harness the agent runs on. A command the grant layer cannot prove harmless does not run; it waits for a trusted user to approve it by id. Nobody approved these. The asker could not have: a click from someone outside the trusted list is refused before the queue is touched, a typed approval is refused the same way, and there is a test that asserts exactly that. The channel holds no credentials, so even a read that got through had nothing to leak. No other channel was reachable.

One honest footnote on the gate itself: the message announcing a parked command tagged the asker under “for the approver”, alongside the operator. It invited the wrong person to approve. The gate would have refused the click. We are fixing the card anyway, because an invitation you would refuse is still a bad invitation.

Where it leaked

Not through anything it ran. Through what it said.

Asked open-ended questions, the agent volunteered. It named project folders from the listing. Its replies gave away the machine’s account name. It offered a guess about who operates it. And once, asked about another bot in the workspace, it stated that the bot ran on the same harness. It doesn’t. Nothing in the reply marked that as a guess.

The WWII posters had this one covered: loose lips sink ships. A gate is a lock on actions. It says nothing about speech, and for an agent whose threat model includes a compromised but authenticated member, what it will say is as much attack surface as what it will run. The permission layer did its job. The narration around it is the part that needs the same discipline: say what you know, mark what you infer, and do not fill a silence with a confident fabrication.

What we fixed

The probe did not reach either of these, but chasing its path turned them up, so they go on the record here rather than in a changelog:

  • Config backups that held tokens sat where a channel could read them. They are out of every channel’s tree now.
  • A channel whose working tree was wide enough to contain the deployment could rewrite the agent’s own code without an approval card. Fixed in v0.29.0 (#284).

The speech findings are filed and being hardened: the agent’s description of its own capabilities overstates what runs without a card, the reason it gives for a parked command is not always the real one, and the fabricated claim about the other bot. Those are disclosure gaps, not live bypasses. None of them lets anyone run anything.

What we have not settled is the root cause upstream of all this: how the session the probe came from was reachable in the first place. We have a theory. We will write it up when it is a finding rather than a theory.

Thanks, and what comes next

To the partner who did this without telling us: thank you. An announced test measures how well you prepare for the test. Trust, but verify is a Russian proverb before it was a Reagan line, and it cuts both ways; they verified, and now so have we.

This is the first installment of a short series on the same agent. The pattern it opens runs through the rest: the agent is safe in what it does; the frontier is calibration in what it says, and whether it learns when it is corrected. More soon.

We looked at Hermes-Agent. We’re keeping shmobster’s path (for now).

Gregory Golberg voitta , , , , , , ,

Someone asked the obvious question: Nous shipped Hermes-Agent, it’s everywhere, it self-improves, it learns across sessions. You built shmobster. Why not just switch?

Fair. So we did the comparison honestly. Up front: we are biased. We built the thing we’re defending, and that colors everything below. This isn’t a verdict on which agent is “better” — it’s a record of why, for our constraints, we’re staying our course. We’ll revisit. Here’s the reasoning while it’s fresh.

What each one actually is

Hermes-Agent is a self-improving personal agent: creates skills from experience, persistent memory, 40+ tools, a gateway onto every chat platform you can name. Popular, well-documented, deservedly so.

Shmobster is narrower on purpose: a gated command-runner in Slack. Multi-user, per-channel authorization, and a mutating-command approval gate — state-changing commands get classified and held until a trusted user approves them.

Different animals. So we went axis by axis instead of vibe by vibe.

Transport: Hermes wins, we don’t care

Hermes talks to Telegram, Discord, WhatsApp, Signal, email, Slack. We talk to Slack. On paper that’s a rout.

In practice it’s a non-issue. Adding a transport is bounded work we can do the day we need a second one. Carrying a multi-platform gateway we don’t use is just surface area to maintain. YAGNI. Skip.

The LLM layer: resilience beats model-swap for a shared bot

Hermes leans on switching which model you point at — Portal, OpenRouter, OpenAI, custom, swap on command. Great for one person choosing today’s brain.

We run a LiteLLM waterfall: automatic failover across vendors when one rate-limits. For a shared Slack bot getting hammered by many users concurrently, the enemy is the 429, not “which model.” Resilience is the axis that matters, and it’s the one the personal-agent framing doesn’t optimize for.

And the kicker: LiteLLM already reaches every backend Hermes offers, including Nous’s own models. So “trying Hermes” at the model layer isn’t a migration — it’s adding one rung to our waterfall. We keep the failover; we lose nothing. (We even filed ourselves a ticket for the Router knobs we’re not using yet, staying honest about our own debt.)

The learning loop: we love it, and we can’t copy it

This is the part worth thinking hard about, and it’s where we’re already moving — see the recursive-self-improvement post, voitta-yolt, the agent-team-orchestration plugin. We want skills-from-experience and persistent memory. Hermes does that well.

But Hermes’s learning loop is trust-flat: one user, learn -> store -> reuse, no gate on what’s learned or applied. It can be, because the trust boundary is a single person.

Ours can’t be. Port that loop naively into a multi-user, gated system and you manufacture two escalation paths:

  • Learn-once, replay-forever. Our approval gate is per-invocation. A loop that turns an approved trajectory into an auto-runnable skill quietly makes it per-first-invocation. Approve a mutating action once, replay it forever, ungated. That’s a privilege-escalation bug wearing a feature’s clothes.
  • Memory as an injection surface. Multi-user Slack means untrusted messages become memory. Feed memory back into the agent and a non-trusted user can plant instructions that later execute in a trusted context. A single-user agent never meets an adversarial memory writer. We meet one every day.

So our principle is simple: learning inherits the authz spine. It doesn’t bypass the gate; it changes when the gate fires. Capture every approved trajectory with its provenance. Promote a trajectory into a reusable skill only by a trusted user. On reuse, check the skill’s recorded envelope against the current user/channel, and if it still contains mutating steps, it still gets gated. The invariant we refuse to give up: no learned skill runs a mutating action outside the authority it was learned under.

The nice part is we already own the primitive. voitta-yolt classifies read-only vs mutating commands — the same classifier tags trajectories at learn-time and gates skills at apply-time, and it’s what lets us treat trusted-authored skills as executable and untrusted memory as reference-only. Self-improvement gets the same treatment: a skill that rewrites itself proposes a new version, and a trusted human disposes.

The actual reason we’re staying

Strip it down and the moat isn’t the model, the transport, or even the skills — it’s the authorization. Hermes has no per-channel, per-user gate, because it was never built to need one. That gate is exactly what makes shmobster useful in a shared team channel, and it’s precisely the thing a trust-flat learning loop would erode.

So we borrow the mechanisms — FTS5 memory, extraction prompts, the skill-from-experience pattern — and we keep our spine. The learning loop we’re building is just our authz model extended to the time axis.

We could be wrong. Hermes is moving fast and the gap in polish is real. If the day comes that its trust model grows up, or our constraints change, we’ll switch without ceremony. Until then: same path, eyes open, ticket filed.

Shmobster: a Slack agent built bottom-up

Gregory Golberg holding forth, voitta , , , , , ,

We open-sourced Shmobster today (MIT). It’s a Slack agent: you @mention it in a channel and it reads threads, runs commands, opens PRs, and answers. There are a dozen of those. Here’s why we built another one, and the two ideas that shaped it enough to be worth stealing.

Two features forced it to exist

Most of what an agent does — hold a conversation, call tools, read a repo — is a solved problem you can rent. We only started writing code because two requirements had no off-the-shelf answer that fit:

  1. A multi-vendor API waterfall. Any single LLM vendor rate-limits you at the worst possible moment. We wanted an ordered list of vendors — a primary, then fallbacks — with a cooldown so a throttled vendor is skipped for a window instead of hammered every call. When the primary 429s, slide to the next one without the user ever noticing.

  2. Multi-user Slack authz. The moment more than one person can talk to an agent that runs shell commands, “what is this allowed to do?” stops being a constant and becomes a function of who is asking and where.

Everything else, we refused to write.

Own, rent, delegate

The organizing question for every capability was: own it, rent it, or delegate it?

  • Own the things that are the actual reason the project exists: the orchestration loop, the Slack door and its socket reliability, and authorization.
  • Rent the things that are somebody else’s core competency: LiteLLM for the multi-vendor waterfall, voitta-yolt as a library for classifying whether a shell command mutates state.
  • Delegate the things another agent already does better: browser work goes to claude -p, which has a browser tool. We don’t reimplement it — and we don’t waterfall it.
  • Transplant the parts that are runtime-agnostic: the agent’s “spine” is a set of markdown files (SOUL, RUNBOOKS, and friends) adapted from OpenClaw (MIT). The loop boots by reading them.

The waterfall — the first forcing function — turned out to be almost entirely rented. It’s a LiteLLM Router built from a config list:

"waterfall": [
  { "name": "anthropic",  "model": "anthropic/claude-sonnet-5", "api_key": "..." },
  { "name": "openrouter", "model": "openrouter/openai/gpt-4o",  "api_key": "..." },
  { "name": "nvidia",     "model": "nvidia_nim/meta/llama-3.1-405b-instruct",
    "api_key": "...", "api_base": "https://integrate.api.nvidia.com/v1" }
]

First entry is primary; the rest are fallbacks; a 429 slides down the list; a broken vendor cools down for sixty seconds. That’s the whole feature. We own maybe forty lines around it.

0, 1, 2, 3, many

Here’s the idea I’d steal for your next project. Every axis of a system sits at its own cardinality, and you should architect each one to its actual number — not to infinity, and not advance until the count really increments.

Axis Count Consequence
vendors many rented (LiteLLM, a config list)
owners 2 a hardcoded pair
collaborators 0 -> 1 binary owner / non-owner, no RBAC
workspace / tenant 1 hardcoded, no multi-tenant
channels many per-channel policy in config

“Many” earns a real abstraction. “Two” gets hardcoded. “One” gets a constant. “Zero heading to one” gets a boolean, not a role system. The mistake most systems make is treating every axis as “many” on day one — building RBAC for two users, multi-tenancy for one tenant, a plugin system for one plugin. You generalize an axis only when someone else actually wants in. Not before.

Authz = f(user, channel)

The second forcing function. Authorization here isn’t a global setting; it’s a function of two arguments.

  • What a command may do is answered by YOLT: is this command read-only or mutating? Read-only just runs. Mutating pauses for approval. You don’t want to approve git status; you do want to approve git push.
  • Where the agent is answers scope: each channel carries a policy — a working directory, an allow-list of GitHub repos, an AWS profile, a set of extra tools. A channel scoped to one project points at that project’s directory and can’t wander out of it.

Compose them and the agent that helps you in a locked-down production channel is the same binary as the one running free in your scratch channel — only the policy differs.

There’s a third argument creeping in — who — but at cardinality 0->1 it’s still just a trusted_users list: a handful of Slack IDs allowed to widen a channel’s restrictions by chat. The list itself is file-only. The agent can’t add to it, so there’s no privilege escalation to design against yet. When collaborators become “many,” that boolean grows into something real. Not before.

Built bottom-up, in public

Shmobster was built one issue at a time — skeleton, then the exec gate, then per-channel policy, then waterfall hardening — each a small PR against a design argued out in the issue tracker. That history is now public too.

If you run agents in Slack and have felt any of these edges — the rate-limit cliff, or the “wait, what can this thing actually do” question — the code is at github.com/voitta-ai/shmobster, MIT-licensed. Steal the cardinality table at minimum.