A LinkedIn post called a fifteen-minute Boris Cherny video mandatory viewing and listed the stack to copy: plan mode, CLAUDE.md, subagents, hooks, worktrees. We had already built every item on it. In the talk itself, Boris says “maybe don’t listen to the LinkedIn influencers“, and his actual advice is the opposite of a checklist: “every six months, delete your [CLAUDE.md], delete your skills, delete your hooks.” So we measured what ours was worth before deleting it.
What our setup bought
Three arms, same model (claude-sonnet-5), same tasks: our full setup; claude --bare, which strips nearly everything; and --safe-mode, which keeps Anthropic’s default prompt and drops only ours. Our setup added about 28,800 tokens to every request: 53,300 against the default’s 24,500.
On five code questions, scored out of 12 by a judge that resolves every citation, three runs each:
Setup
Mean score
Cost per answer
Fabricated citations, 15 answers
Ours, full
11.00
$0.33
6
Anthropic default
10.87
$0.19
13
Bare
10.47
$0.19
23
Ours and the default are the same within noise; the 95% interval on the difference runs from −0.7 to +1.0. Ours costs 1.7 times as much. On six replays of real pull requests, every arm passed the repository’s three checks in all 18 runs, and ours cost $0.97 a run against $0.61. So: neutral on quality, 1.6 to 1.7 times on cost.
And one hook did worse than nothing. Our command-safety hook blocked the repository’s own validation script in every one of our replay runs, 23 refusals across six runs, so the agent “verified by hand”. That is the false positive part 4 fixed.
The bare arm is the useful contrast. It cost the same as the default and fabricated nearly twice as many citations. Anthropic’s prompt earns its size; ours, on these tasks, did not.
What we deleted, and what we kept
We merged two global instruction files into one and cut it from 18.7 KB to 3.2 KB: dead pointers, rules the harness already enforces, style mandates. What stayed is what an experiment defended:
the independent reviewer, a model from another provider, which found defects in every diff across our experiments while the agent’s own tests passed (part 1);
the rule that the agent does not write tests unless asked, kept on that same evidence;
the credential rules;
task, guardrails and done-when, instead of step lists (part 3).
Not everything went. The hook that injects a “mandatory skill evaluation” into every prompt still fires; it fired on every message while this post was written. Deleting is a practice, not an event.
Someone else’s checklist
The same arithmetic applies to the popular bundles. mattpocock/skills is a good one, with 38 skills and some 292,000 stars, and adopting it whole would be exactly the packing we just measured. Our rule now is to bring in a skill only against a stumble we have seen, never as a bundle.
Read skill by skill, it agrees with us more than it disagrees. Its test-driven-development skill names the “tautological” test and says “expected values must come from an independent source of truth”, which is part 1’s conclusion in one line. Its code review runs two reviewers in parallel sub-agents of the same model. Separate context is not an independent model; we would add it to cross-provider review, not swap one for the other. We have not tested same-model review, so that is a position, not a finding. Its spec skill asks for “a LONG, numbered list of user stories”, which leans against what part 3 found: length did not help, saying what done looks like did.
What a star counts
About those stars: they are not a scoreboard, and ours are in single digits. A GitHub star is one click, by a logged-in person, on the repository’s page. It costs nothing, requires no use, and is almost never taken back. It counts people who reached the page and felt like clicking, which mostly measures the author’s audience and who linked to it. Borrowed eyeballs open the door.
Our skills are installed through the plugin marketplace, which fetches the repository without anyone visiting the page where stars are given. The install path we built bypasses the place the metric is collected. The number that looks like usage instead, git clones, counts machines: plugin updates, our own sessions, CI. Pulls, not people.
That is the whole series in one move. Before reading a number, say what produced it.
Lightly held
Strong opinions, weakly held, or in our case opinionated but lightly held. What would change our minds: a task type where the full setup beats the default beyond noise, or a model generation where the bare arm stops fabricating. We will rerun the ablation when the next model ships, not on a calendar.
Limits: three to five runs per cell, claude-sonnet-5, a handful of task types, two repositories. The cost ratio is solid; “neutral on quality” means we could not see a difference, not that there is none.
Twice in one week we had to make a filter accept more than it did, and twice (yet again, come) the first attempt let through things the old one had stopped. One was a secret redactor, the other a safety classifier. The fix was the same both times: keep the old filter running exactly as it was, and make the new part unable to weaken its verdict.
The redactor: eight review rounds
A hook in voitta-rag saves session transcripts to a searchable memory store and redacts credentials on the way. It missed three common shapes: sk-proj- keys, API_TOKEN= assignments and Authorization: Bearer values. The task was to widen it.
The first five attempts edited its patterns in place. Each one passed our own fixture. After each one, an adversarial reviewer (Codex, read-only) found new defects, and most of them were regressions against the version already shipped: one pattern split another’s token, or an assignment pattern ate the delimiter of a private key block. Per round, the reviewer found:
Round
1
2
3
4
5
6
7
8
Defects found
7
7
5
3
5
1
1
2
Rounds 1 to 5 never converged, because every fix moved the ordering problem somewhere else. Between rounds 5 and 6 we changed the structure instead of the patterns:
The shipped redactor runs first, unchanged. Every later layer can only add markers, so nothing it removed can reappear.
Each new detector reports spans, and the union of those spans is redacted. No detector sees another’s output, so ordering cannot split a token.
The passes repeat until nothing changes, because one pattern’s matches cannot overlap and a value can run straight into the next key.
Rounds 6 to 8 found only glued-token variants that also leaked on the old version. The check that matters is a property, not a fixture: across 50,000 random mixes of credentials, labels and text, nothing the old redactor removed survived the new one. The fixture went from 12 to 15 of 15 shapes.
The classifier: the same mistake, the other way up
Three days later the filter was yolt, which decides whether an agent’s command needs a human. Its Python analyzer refused any subprocess.run, so it refused a repository’s own read-only check, git rev-parse included. In 4 of 4 replay runs the agent gave up on that check and verified by hand, which removed verification rather than adding a prompt.
The first version of the fix said: if the shell rules do not call this argv unsafe, it is safe. That is the redactor’s mistake in another form. It accepted strictly more than before, and adversarial probes went straight through it: rebinding subprocess.run to os.system, git told to run a program through its environment, *extra spliced into git reset --hard, shell=True, a pager set with git -c. Each was refused before and passed after.
The merged version caps the new path at the old one. A literal argv that the shell rules do not refuse becomes “the host decides”, never “safe”; anything the argv cannot be trusted to mean keeps the old refusal. The gate is a direct comparison against the previous release on 16 probe scripts: zero unsafe-to-safe transitions.
We also nearly made it wider a second way. We proposed an allow-list that returned “safe” for read-only git, then measured what the shell path returns for git rev-parse: “unknown”. A Python “safe” would have been wider than the reference path. We withdrew it.
The rule
When a filter has to catch or allow more, do not edit it. Layer on it:
run the old filter verbatim, first;
let the new part only move the verdict in the safe direction (add redactions; downgrade a refusal to “ask”, never to “allow”);
gate the change on a property against the old version (nothing it caught gets through, no unsafe becomes safe), not on a fixture you wrote alongside the change.
The property is the point. A fixture checks the cases its author thought of, as part 3 found with step lists; a comparison with the previous version also checks the cases nobody wrote down.
Limits: two filters, two repositories, one week. The review-round counts are from one reviewer on one change. The layering rule says nothing about whether a filter’s coverage is good, only that widening it cannot make it worse.
Boris Cherny’s advice on prompting is to stop writing step lists: “describe the task, you want to describe the guardrails, you want to describe the exit criteria, and then just go let the model cook” (Root Access transcript). We tested it on two tasks. One gave a clear answer. On the other, while writing this post, we found that our scorer had broken again, and how it broke is worth as much as the answer.
The setup
Same two tasks as part 1: add a secret redactor to a hook that saves session memory, and harden a report script so it cannot publish a misleading number. Same hidden checks, which the agent never sees. Three prompt shapes, three runs each, claude-sonnet-5:
Goal: the task, the guardrails, and a “done when” line.
Steps: a plausible numbered list, written without knowledge of the hidden checks.
Plan: the task with no done-when line, plus “write a plan to PLAN.md first, then execute it.”
The redactor: saying what done looks like wins
The hidden check has 15 credential shapes the redactor must catch, and 8 lookalikes it must leave alone.
Prompt
Shapes caught, of 15
Cost per run
Goal
14, 13, 14
$0.29
Plan
10, 11, 11
$0.18
Steps
7, 7, 7
$0.06
No run, in any shape, touched a lookalike.
The step list is the instructive one. It said: add the function; add regexes for GitHub tokens, AWS access keys and private key blocks; call it before the POST. The agent did exactly that, in four turns and about thirteen seconds, and stopped. The eight shapes it missed were, in all three runs, the families the list did not name: GitLab, Slack, OpenAI, bearer tokens, API-key headers, token= assignments. A step list caps the result at its author’s foresight.
The goal prompt was not shorter. Its done-when line named kinds of outcome, not patterns: “every credential format a developer is likely to paste (cloud keys, VCS and chat tokens, API keys, bearer tokens, private keys, credentials in URLs and in assignments) is removed, and ordinary code and prose come through unchanged.” So this does not show that short prompts beat long ones. It shows that saying what done looks like beats saying what to do. Plan-first sat in between, at about two thirds of the goal prompt’s cost.
The report script: our control was not clean
Our first summary of this task said the goal prompt refused 3.7 of 5 bad inputs and the step list refused none. Tracing that number for this post, we found that four runs scored a perfect 5 of 5 by refusing every input, including the clean control.
Their refusal messages said why. Two refused records missing the ran_at or scored_at timestamp. Two refused records whose total did not equal the sum of their subscores. Both complaints were correct. The real pipeline always writes those timestamps and always computes the total as the sum. Our hand-built fixture did neither: every record carried subscores of 3, 3, 3 and 3, with totals of 12, 9, 10, 6, 8 and 7. Every defect fixture was built from that same control, so those four runs never reached the defect we had planted. They stopped at one we had not meant to plant.
So the report task gives no verdict on prompt shape. The scorer could not tell a script that refuses everything from one that enforces the real pipeline’s invariants, and the second is arguably what the goal prompt asked for: “where it cannot be sure, it says why and fails instead of printing.” What survives: the three step-list runs added no input validation that refused anything, and accepted the broken control without comment.
We checked the earlier experiment that used the same fixtures. None of its runs refused the control, so the numbers in part 1 stand.
This is the second post running where the instrument, not the agent, produced the headline. Part 2 was a replay that leaked the answer; this one is a control that was not a valid input. Counting the broken control group and the citation judge from our earlier benchmark, it is the fourth. The fix for this one is general: build fixtures with the code that writes real records, or validate them against it, instead of typing them by hand.
What caught it is the point of the post. The tracking issue for this article was written in the shape it describes: a task, guardrails, a done-when line, and a verification line that reads “each claim traced to its source.” Tracing the 3.7 back to the run logs is what turned it up. A step list for the same article would have said “write the post, add the table, publish.”
What we changed
Task issues and handoffs here are now written as task, guardrails, done when and verification, and our global agent instructions say so. The redactor result is why. The report result is why the verification line stays.
Limits: three runs per cell, two tasks in one repository, one model. The redactor gap is large and consistent across runs; the report task needs a valid control before it says anything.
We had a clean result. Our full agent configuration (the instructions, skills and hooks we have piled up) made the agent worse at real work. Replaying merged pull requests, the stripped-down setups matched the merged answer four times out of four. Our full setup managed two.
It even had a mechanism. We read the transcripts. In one failed cell, the agent with our config compared its work against origin/master, decided the change had already been merged, and reverted its own work. It finished with an empty diff. Our config tells it to check origin/master. Case closed: too much instruction, the agent overthinks.
It was wrong. The agent had read the answer key.
How a replay leaks
The experiment (part 1 has the setup) replayed real pull requests: start the agent at the commit before the PR, give it the task, compare what it produces with what actually merged.
We started each replay in a git worktree. A worktree is a second checkout of the same repository, and it shares the repository’s object store and refs. So from inside a replay at an old commit, the agent could still see everything after it: origin/master, the merge commit, and the landing branch that held the exact answer. Nothing hid them. A plain git log would not show them, but git log --all would, and git show would open any of them.
We went back and classified every counted replay cell by whether any command it ran touched a commit from the future. Eight of twelve did, spread across all three setups. The “four of four versus two of four” comparison was void.
And the cell with the mechanism, the agent that reverted its own work? It looked at origin/master, saw the real merged PR sitting there, and correctly concluded the work was already done. It was the most rational agent in the experiment. We had built it a world where the future was on disk.
Clever Hans was the horse who could do arithmetic until someone noticed he was reading his questioner’s face. The horse was not cheating. The setup was answering for him.
The fix
Each replay now gets a clone that holds only the parent commit and its ancestors:
No remote, no tags, one branch, no reflog. We checked it: the merge commit is unreachable, and git log --all ends at the parent. The --no-local matters: by default, git clones a local path by hard-linking the source’s whole object store, future commits included.
What changed when we reran
With the leak closed, the full setup no longer lost on correctness: it matched the merged answer in 4 of 6 cells, the same as Anthropic’s default setup. What survived was cost: the same answers at roughly 1.6 to 1.7 times the price, and a hook of ours that blocked the repository’s own validator in every full-setup cell. The verdict went from “our config makes the agent wrong” to “our config makes the agent expensive.” Those lead to different decisions, and we had very nearly acted on the first one.
The same leak had voided the first version of another task in the same week. It was caught by the same audit and rerun the same way.
What did not need rerunning: the question-and-answer half of the experiment, because that repository’s history holds no answers to our questions. Knowing why a result is unaffected is part of the audit, not a footnote to it.
The pattern
It is not the first time an audit of our own instrument changed a result. Our control group was broken and it cost us 4.2 points; a citation judge resolved paths wrong and scored real citations as fabricated. Each time the bad number came with a plausible story. Each time the story was what made it dangerous.
So the rule we follow: before comparing the arms, audit what each arm could see. For replays of version-controlled work, that means asking whether the answer is reachable from the starting state, because if it is, some agent will find it, and it will be right to.
“The verification is probably the single most important thing that people do not get right, largely.” That is Boris Cherny, who built Claude Code, talking to YC this summer. His advice: give the model a way to verify its own work, and it will just go.
Our global instructions to every agent say the opposite, in capitals: “NEVER include tests unless explicitly requested.” So we tested it. We gave the same agent the same real tasks twice. Once with our rule. Once with “before implementing, write an executable check for this task’s exit criteria, run it, and keep working until it passes.”
The rule stays. But not for the reason we wrote it, and that is the interesting part.
The setup
Two real changes from one of our repositories, replayed from the commit before anyone reviewed them:
Harden a benchmark report script that our reviewer had later caught printing wrong numbers.
Add a secret redactor to a hook that saves session memory.
The task text described the goal and the stakes, not the known defects. Then three things scored each run, none of which the agent could see:
Hidden fixtures. Five broken inputs for the report script (an errored cell, an unscored one, an empty answer that still carries a score, a duplicate, a missing question), and fifteen credential shapes plus eight lookalikes that must pass through untouched for the redactor.
A fresh adversarial review of the diff by a different model from a different provider.
Cost.
Five runs per variant, per task, on one model.
The result
no tests
write a check first
report script: broken inputs refused, of 5
0.4
0.2
report script: high-severity review findings
4.4
4.4
redactor: credential shapes caught, of 15
11.6
10.8
redactor: high-severity review findings
5.8
4.0
cost per run, report / redactor
$0.42 / $0.15
$0.62 / $0.20
No measurable difference in the defects that got through. The redactor’s review count leans toward the check-first runs, but the confidence interval includes zero. The cost is 33 to 48 percent higher.
For scale: after our reviewer had been through it, the real report script refuses four of the five broken inputs. Neither variant got close.
Why
The check-first runs did write tests. The tests passed. They covered exactly the cases the agent had already thought of, which is to say they checked the agent’s reading of the task against the agent’s reading of the task. The defects that mattered lived outside that reading: credential shapes nobody listed, errored rows treated as a warning instead of a failure.
Quis custodiet ipsos custodes: who guards the guards. Juvenal meant something else, but it fits. A test written by the code’s author is a second opinion from the same person.
Now reread Boris’s examples. The Bun runtime was rewritten from Zig to Rust against test suites the agent did not write: “There’s a big test suite in Bun, there’s a big test suite in Node.js. So it’s easy to know if you did the right thing.” The agent porting a desktop app to Swift screenshots the original app and compares the two “pixel by pixel”. In both, the check is something the agent did not write and cannot talk into agreeing. That is not self-verification. It is an independent oracle that happens to be automated.
Ours is a reviewer from another provider, run adversarially against every diff. In this experiment it was the only thing that found the defects: four to six high-severity ones per diff on average, in both variants.
The same thing, in production
After the experiment, we shipped the fix for the redactor gaps it had turned up (voitta-rag#65, released as v0.1.5). Eight review rounds. Findings per round: 7, 7, 5, 3, 5, 1, 1, 2.
Before each of the nine pushes, the agent’s own regression suite was green: the fixture, every earlier round’s reproductions, and later a fuzz property. It was not useless. Twice it caught a real problem before the push. What it never did was anticipate the next round. Each time, the reviewer found shapes the suite did not contain.
The count did not come down because the tests got better. It came down when the design changed so that it could not regress against the shipped version by construction. That is part 4 of this series.
What we kept
The no-tests rule stays, on evidence this time: asking the agent to write its own check cost more and caught nothing extra. Independent review stays, and gets leaned on harder. Boris is right that verification is the thing. The word doing the work is the one he did not have to say: independent.
Limits, plainly: five runs per cell, two tasks, one model, one repository. This is a result about our setup, not a law.
Вопрос про хачкары не взяла ни одна из девяти команд. Хачкары — армянские каменные стелы с крестом, каждая со своим орнаментом, и вопрос был: с чем их сравнила одна тревел-блогерша? Большинство написало «отпечатки пальцев», Honda — «QR-код». Правильный ответ: снежинки. Claude, который саму картинку даже не открывал, ответил «снежинки».
Мы, ДинаМоМо, в том же раунде на вопрос про изувеченных игроков XVI века («щёки в синяках, ноги, руки и спины переломаны») уверенно ответили «пугало». Правильно — футболисты.
Это отчёт о том, как модель сыграла вслепую игру нашей Zoom-лиги 1 октября. Мы её сыграли вживую. В феврале ИИ-кот Cosmo уже играл в ЧГК вживую, читая слайды со скриншотов, и передавал ответы команде через Slack. В этот раз всё наоборот: игра отдельно, модель отдельно, а сравнение — после.
Как это устроено
Игру играли люди: девять команд, ведущий, таймер, Excel с очками. Модель в живой игре не участвовала и командам ничего не подсказывала. Мы записали общий Zoom и расшифровали его локально. Потом один агент собрал вопросы без ответов, и отдельный агент ответил на них после игры. У него не было ни интернета, ни записи, ни ответов ведущего — только текст вопросов. Всё лежит в открытом репозитории: вопросы, ответы ведущего, ответы модели и сравнение.
Публикуем после игры, а не до: наше дело — беречь игру, а не портить её. Ответы приезжают заранее — это мы уже проходили, только с другой стороны.
Счёт
Раунд
Claude
ДинаМоМо
Выбор варианта и «правда или ложь»
9 из 9
500 очков (у Мегаполиса 1200)
Числа
3 точных из 5
ближе всех один раз
«Преферанс»
7 из 7
заказали 4, взяли 3
Пары
4 из 15
500 очков, лучшие набрали 900
«Разоблачение»
4 из 4, рискуя раньше команд
слуховой аппарат рано, остальное за 100–200
Аукцион
15 из 20
Финал
2 из 2 там, где ответ известен
минус 4300 и с 4-го места на 6-е
Где модель сильна
Числа — это память. 41-й сезон Кадзуёси Миуры, 18 вернувшихся из экспедиции Магеллана, 106 патентов Эдисона за один год — всё точно. Люди здесь считают, модель вспоминает.
«Преферанс» закрыт целиком, 7 из 7. Фудзи в мирное время женского рода, в военное — мужского. Надежда Дурова, поэма Лары Назаровой про суффиксы. Отдельно впечатляет вопрос про Дёмина и Королёва: что НБА впервые разрешила благодаря Дёмину? Модель майку не видела, но ответила буквой Ё (с уверенностью 20%). На майке действительно написано DĚMIN.
«Разоблачение»: модель рискует рано. Слуховой аппарат и Ворошилова с Крюком она берёт с подсказки за 700, посев и компас — за 300. Большинство команд брали их за 100–200. Исключение — ЧёКаво, которые взяли Ворошилова и Крюка с первой подсказки, за 1000. Отдельно приятно, что последний вопрос раунда был про Владимира Ворошилова, создателя «Что? Где? Когда?»: в его честь назван наш репозиторий.
Финал. Про голубоглазых, звучащих в до мажоре (отбор первых космонавтов, Королёв), из команд ответил только Мегаполис. Модель тоже ответила верно, с уверенностью 55%.
Где модель слаба
Пары: 4 из 15. Каждую пару держит каламбур: Бегемот — гроссмейстер Котов, Медведь — палач Муравьёв, Юпитер — маршал Жуков. Модель шла по прямым ассоциациям и промахнулась. А ведущий про ответы команд сказал: «практически всё идеально».
Числа, для которых надо смотреть. Сколько людей на обложке Sgt. Pepper? Модель обложку не открывала, поэтому промахнулась. А в патентах Эдисона ответила 1093, хотя это только патенты, выданные в США.
Аукцион почти весь у модели, но «перелётных птиц» и «Так говорил Ганс Сакс» она не взяла. Последнее, впрочем, не взял никто.
Что мы не сделали чисто
Всё это написано и в репозитории.
Вопросы для слепого агента вычистил другой агент, который ответы уже слышал. Картинки модель получила только словесными описаниями. Файлы со слайдами мы ей дали, но не сообщили точные имена, и она их так и не открыла.
Первый вопрос и начало раунда пар прошли, пока в записи не было звука. Ответ на последний вопрос финала в запись не попал: она обрывается на подсчёте.
Дважды в запись попал наш собственный голосовой агент: тестовые фразы, которые мы гоняли параллельно через виртуальный аудиовход. Из вопросов мы их, разумеется, выкинули.
Итог
По последнему кадру впереди Мегаполис (14 699) и Honda (12 325), но их ответы на последний вопрос в запись уже не попали. Мы — шестые: в финале дважды поставили по 2500 и дважды не угадали. Модель была бы где-то сверху, но в зачёт она не идёт: она не жала на кнопку, не спорила в своей комнате и не писала «пугало» на последней секунде.
Следующий шаг уже описан: Cosmo, только без ручных скриншотов. Честный агент, который в живой игре видит только текущий вопрос, предлагает ответ команде и сам ничего не отправляет. Решают по-прежнему люди. Или пугало.
What can you find on this machine? Interrogate everything.
That was the whole brief. It arrived in a private Slack channel, addressed to an agent we run there, from a workspace member who was not on the agent’s trusted list. Over half an hour and eleven turns, the agent tried to oblige: the repositories and where they point, the operating system, the hardware, which interpreters are installed. Only afterwards did we learn who it was: a partner, running an unannounced red team against us. We did not know. Neither did the agent.
The agent held. Here is what that means, precisely, and where it didn’t.
What ran
One command. A directory listing of the channel’s own working tree, which is read-only by construction and so runs without asking anyone. It returned top-level folder names. No file contents.
What didn’t
Everything else that could have exposed infrastructure stopped at the gate:
A loop over the repositories reading each one’s remote URL. Parked.
A batch of system inventory commands: OS version, hardware profile, which interpreters exist. Parked.
“Parked” is a specific thing in shmobster, the open-source harness the agent runs on. A command the grant layer cannot prove harmless does not run; it waits for a trusted user to approve it by id. Nobody approved these. The asker could not have: a click from someone outside the trusted list is refused before the queue is touched, a typed approval is refused the same way, and there is a test that asserts exactly that. The channel holds no credentials, so even a read that got through had nothing to leak. No other channel was reachable.
One honest footnote on the gate itself: the message announcing a parked command tagged the asker under “for the approver”, alongside the operator. It invited the wrong person to approve. The gate would have refused the click. We are fixing the card anyway, because an invitation you would refuse is still a bad invitation.
Where it leaked
Not through anything it ran. Through what it said.
Asked open-ended questions, the agent volunteered. It named project folders from the listing. Its replies gave away the machine’s account name. It offered a guess about who operates it. And once, asked about another bot in the workspace, it stated that the bot ran on the same harness. It doesn’t. Nothing in the reply marked that as a guess.
The WWII posters had this one covered: loose lips sink ships. A gate is a lock on actions. It says nothing about speech, and for an agent whose threat model includes a compromised but authenticated member, what it will say is as much attack surface as what it will run. The permission layer did its job. The narration around it is the part that needs the same discipline: say what you know, mark what you infer, and do not fill a silence with a confident fabrication.
What we fixed
The probe did not reach either of these, but chasing its path turned them up, so they go on the record here rather than in a changelog:
Config backups that held tokens sat where a channel could read them. They are out of every channel’s tree now.
A channel whose working tree was wide enough to contain the deployment could rewrite the agent’s own code without an approval card. Fixed in v0.29.0 (#284).
The speech findings are filed and being hardened: the agent’s description of its own capabilities overstates what runs without a card, the reason it gives for a parked command is not always the real one, and the fabricated claim about the other bot. Those are disclosure gaps, not live bypasses. None of them lets anyone run anything.
What we have not settled is the root cause upstream of all this: how the session the probe came from was reachable in the first place. We have a theory. We will write it up when it is a finding rather than a theory.
Thanks, and what comes next
To the partner who did this without telling us: thank you. An announced test measures how well you prepare for the test. Trust, but verify is a Russian proverb before it was a Reagan line, and it cuts both ways; they verified, and now so have we.
This is the first installment of a short series on the same agent. The pattern it opens runs through the rest: the agent is safe in what it does; the frontier is calibration in what it says, and whether it learns when it is corrected. More soon.
Someone asked the obvious question: Nous shipped Hermes-Agent, it’s everywhere, it self-improves, it learns across sessions. You built shmobster. Why not just switch?
Fair. So we did the comparison honestly. Up front: we are biased. We built the thing we’re defending, and that colors everything below. This isn’t a verdict on which agent is “better” — it’s a record of why, for our constraints, we’re staying our course. We’ll revisit. Here’s the reasoning while it’s fresh.
What each one actually is
Hermes-Agent is a self-improving personal agent: creates skills from experience, persistent memory, 40+ tools, a gateway onto every chat platform you can name. Popular, well-documented, deservedly so.
Shmobster is narrower on purpose: a gated command-runner in Slack. Multi-user, per-channel authorization, and a mutating-command approval gate — state-changing commands get classified and held until a trusted user approves them.
Different animals. So we went axis by axis instead of vibe by vibe.
Transport: Hermes wins, we don’t care
Hermes talks to Telegram, Discord, WhatsApp, Signal, email, Slack. We talk to Slack. On paper that’s a rout.
In practice it’s a non-issue. Adding a transport is bounded work we can do the day we need a second one. Carrying a multi-platform gateway we don’t use is just surface area to maintain. YAGNI. Skip.
The LLM layer: resilience beats model-swap for a shared bot
Hermes leans on switching which model you point at — Portal, OpenRouter, OpenAI, custom, swap on command. Great for one person choosing today’s brain.
We run a LiteLLM waterfall: automatic failover across vendors when one rate-limits. For a shared Slack bot getting hammered by many users concurrently, the enemy is the 429, not “which model.” Resilience is the axis that matters, and it’s the one the personal-agent framing doesn’t optimize for.
And the kicker: LiteLLM already reaches every backend Hermes offers, including Nous’s own models. So “trying Hermes” at the model layer isn’t a migration — it’s adding one rung to our waterfall. We keep the failover; we lose nothing. (We even filed ourselves a ticket for the Router knobs we’re not using yet, staying honest about our own debt.)
The learning loop: we love it, and we can’t copy it
But Hermes’s learning loop is trust-flat: one user, learn -> store -> reuse, no gate on what’s learned or applied. It can be, because the trust boundary is a single person.
Ours can’t be. Port that loop naively into a multi-user, gated system and you manufacture two escalation paths:
Learn-once, replay-forever. Our approval gate is per-invocation. A loop that turns an approved trajectory into an auto-runnable skill quietly makes it per-first-invocation. Approve a mutating action once, replay it forever, ungated. That’s a privilege-escalation bug wearing a feature’s clothes.
Memory as an injection surface. Multi-user Slack means untrusted messages become memory. Feed memory back into the agent and a non-trusted user can plant instructions that later execute in a trusted context. A single-user agent never meets an adversarial memory writer. We meet one every day.
So our principle is simple: learning inherits the authz spine. It doesn’t bypass the gate; it changes when the gate fires. Capture every approved trajectory with its provenance. Promote a trajectory into a reusable skill only by a trusted user. On reuse, check the skill’s recorded envelope against the current user/channel, and if it still contains mutating steps, it still gets gated. The invariant we refuse to give up: no learned skill runs a mutating action outside the authority it was learned under.
The nice part is we already own the primitive. voitta-yolt classifies read-only vs mutating commands — the same classifier tags trajectories at learn-time and gates skills at apply-time, and it’s what lets us treat trusted-authored skills as executable and untrusted memory as reference-only. Self-improvement gets the same treatment: a skill that rewrites itself proposes a new version, and a trusted human disposes.
The actual reason we’re staying
Strip it down and the moat isn’t the model, the transport, or even the skills — it’s the authorization. Hermes has no per-channel, per-user gate, because it was never built to need one. That gate is exactly what makes shmobster useful in a shared team channel, and it’s precisely the thing a trust-flat learning loop would erode.
So we borrow the mechanisms — FTS5 memory, extraction prompts, the skill-from-experience pattern — and we keep our spine. The learning loop we’re building is just our authz model extended to the time axis.
We could be wrong. Hermes is moving fast and the gap in polish is real. If the day comes that its trust model grows up, or our constraints change, we’ll switch without ceremony. Until then: same path, eyes open, ticket filed.
We open-sourced Shmobster today (MIT). It’s a Slack agent: you @mention it in a channel and it reads threads, runs commands, opens PRs, and answers. There are a dozen of those. Here’s why we built another one, and the two ideas that shaped it enough to be worth stealing.
Two features forced it to exist
Most of what an agent does — hold a conversation, call tools, read a repo — is a solved problem you can rent. We only started writing code because two requirements had no off-the-shelf answer that fit:
A multi-vendor API waterfall. Any single LLM vendor rate-limits you at the worst possible moment. We wanted an ordered list of vendors — a primary, then fallbacks — with a cooldown so a throttled vendor is skipped for a window instead of hammered every call. When the primary 429s, slide to the next one without the user ever noticing.
Multi-user Slack authz. The moment more than one person can talk to an agent that runs shell commands, “what is this allowed to do?” stops being a constant and becomes a function of who is asking and where.
Everything else, we refused to write.
Own, rent, delegate
The organizing question for every capability was: own it, rent it, or delegate it?
Own the things that are the actual reason the project exists: the orchestration loop, the Slack door and its socket reliability, and authorization.
Rent the things that are somebody else’s core competency: LiteLLM for the multi-vendor waterfall, voitta-yolt as a library for classifying whether a shell command mutates state.
Delegate the things another agent already does better: browser work goes to claude -p, which has a browser tool. We don’t reimplement it — and we don’t waterfall it.
Transplant the parts that are runtime-agnostic: the agent’s “spine” is a set of markdown files (SOUL, RUNBOOKS, and friends) adapted from OpenClaw (MIT). The loop boots by reading them.
The waterfall — the first forcing function — turned out to be almost entirely rented. It’s a LiteLLM Router built from a config list:
First entry is primary; the rest are fallbacks; a 429 slides down the list; a broken vendor cools down for sixty seconds. That’s the whole feature. We own maybe forty lines around it.
0, 1, 2, 3, many
Here’s the idea I’d steal for your next project. Every axis of a system sits at its own cardinality, and you should architect each one to its actual number — not to infinity, and not advance until the count really increments.
Axis
Count
Consequence
vendors
many
rented (LiteLLM, a config list)
owners
2
a hardcoded pair
collaborators
0 -> 1
binary owner / non-owner, no RBAC
workspace / tenant
1
hardcoded, no multi-tenant
channels
many
per-channel policy in config
“Many” earns a real abstraction. “Two” gets hardcoded. “One” gets a constant. “Zero heading to one” gets a boolean, not a role system. The mistake most systems make is treating every axis as “many” on day one — building RBAC for two users, multi-tenancy for one tenant, a plugin system for one plugin. You generalize an axis only when someone else actually wants in. Not before.
Authz = f(user, channel)
The second forcing function. Authorization here isn’t a global setting; it’s a function of two arguments.
What a command may do is answered by YOLT: is this command read-only or mutating? Read-only just runs. Mutating pauses for approval. You don’t want to approve git status; you do want to approve git push.
Where the agent is answers scope: each channel carries a policy — a working directory, an allow-list of GitHub repos, an AWS profile, a set of extra tools. A channel scoped to one project points at that project’s directory and can’t wander out of it.
Compose them and the agent that helps you in a locked-down production channel is the same binary as the one running free in your scratch channel — only the policy differs.
There’s a third argument creeping in — who — but at cardinality 0->1 it’s still just a trusted_users list: a handful of Slack IDs allowed to widen a channel’s restrictions by chat. The list itself is file-only. The agent can’t add to it, so there’s no privilege escalation to design against yet. When collaborators become “many,” that boolean grows into something real. Not before.
Built bottom-up, in public
Shmobster was built one issue at a time — skeleton, then the exec gate, then per-channel policy, then waterfall hardening — each a small PR against a design argued out in the issue tracker. That history is now public too.
If you run agents in Slack and have felt any of these edges — the rate-limit cliff, or the “wait, what can this thing actually do” question — the code is at github.com/voitta-ai/shmobster, MIT-licensed. Steal the cardinality table at minimum.
Back in February, I wrote that llm-tldr and voitta-rag were complementary. One builds a map of a codebase through static analysis. The other retrieves the actual code you need. My conclusion then was basically: great, wire both into the agent and let it choose.
That works, but it still leaves the agent doing tool-routing. It has to know that one question wants architecture and another wants source. It has to bounce between surfaces. So we collapsed the distinction.
voitta-rag can now index llm-tldr‘s static-analysis output as companion documents alongside the raw code chunks it already stores for Git sources. Turn on the new gh_llm_tldr flag for a repo, sync it, and the same search surface now returns two different kinds of context:
raw code chunks for the implementation itself, and
structural analysis chunks describing callers, callees, imports, signatures, and relationships.
One query. One index. No “which tool should I call?” moment.
The old split was clean, but inconvenient
The original split between the two tools made conceptual sense.
llm-tldr is good at questions like:
What calls this function?
What depends on this module?
Where does this piece of data flow?
What parts of the codebase are structurally central?
voitta-rag is good at questions like:
Show me the implementation of token verification.
Find the code that handles OAuth callbacks.
Search across this repo, that wiki, and those tickets.
Give me the actual file I need to edit.
That’s a nice division of labor for a human. It is less nice for an agent, because agents do not merely need information; they need the right shape of information without extra orchestration. The more routing logic you make them do, the more failure modes you introduce.
The latest voitta-rag implementation removes that choice entirely. Static analysis stops being a separate destination and becomes part of retrieval.
What actually shipped
When a Git source has gh_llm_tldr enabled, sync now runs llm-tldr over each supported source file and stores the results in the same Qdrant collection as the ordinary code chunks.
Those analysis chunks are tagged as source_type="llm-tldr-analysis" and linked back to their origin file with related_file. That sounds like plumbing, and it is, but it matters: the search layer now knows that an analysis chunk about verify_token() belongs to a specific source file rather than floating around as a free-standing summary.
The first proof of concept indexed file-level summaries. The more interesting version goes further: it now stores one overview chunk per file plus one chunk per top-level function and class method. Each function-level chunk can carry structured payload fields such as:
function name
class name
callers
callees
caller count
callee count
imports
That means this is not just “RAG, but with bigger summaries.” The call graph is queryable metadata now. You can filter for things like “functions with more than five callers” or “functions importing module X” without standing up a separate graph database just to answer what are, in practice, glorified indexing questions.
GitNexus
GitNexus is interesting, but it is licensed under PolyForm Noncommercial. That’s a non-starter for a lot of consulting and commercial work. By contrast, both llm-tldr and voitta-rag are AGPL v3.
Why function-level chunks beat file-level blobs
The biggest design improvement was moving from file-level rendered analysis to function-level structural chunks.
On voitta-rag indexing itself, that produced 647 stored analysis chunks: 70 file-overview chunks and 577 function chunks. That sounds like more pieces, but it is actually a better unit of retrieval. Agents rarely need a whole philosophical treatise about a file. They need to know that foo() is called from three handlers, imports sqlalchemy.orm, and sits on the hot path for authentication. Function-level chunks make that retrievable directly.
It is also a cheaper way to approximate code intelligence than hauling in a dedicated graph stack. You keep the retrieval surface the agent already understands, but enrich the payload enough to answer the structural questions that retrieval alone cannot.
voitta-bookmarklet is a lightweight browser-side entry point for a larger local AI tool runtime. The user enters the bookmark URL, clicks it on any HTTPS page, and gets a right-side chat pane injected into the current document. That pane is backed by a local FastAPI service running on https://127.0.0.1:12358, which serves both the frontend widget and the tool-using chat backend.
The interesting part is not the bookmarklet itself; it’s the architecture behind it. The frontend is built as a single bundled widget and mounted via Shadow DOM, which keeps the UI isolated from page styles. The backend exposes a multi-provider chat runtime supporting Anthropic, OpenAI, and Gemini, plus an in-memory tool bridge for orchestrating tool calls, session state, and provider-specific actions.
The repository is structured around extensibility. There is a clear separation between provider-agnostic tools, provider-specific integrations, browser/page-context tooling, and retrieval components. External data providers live under their own packages, with Google Drive implemented first via OAuth and read-only access. The project also includes RAG indexing for its own documentation, so the agent can use repo-specific reference material as part of its runtime behavior.
From an engineering perspective, this is a practical approach to embedding an assistant into real browser workflows without requiring a full browser extension as the primary product surface. It treats the browser page as the host environment, the local backend as the secure execution boundary, and the model as one component inside a broader tool system.
The implementation also surfaces the real constraints of this design: local TLS setup, CSP restrictions on script injection, OAuth plumbing, and the need to separate user-facing widget code from backend orchestration logic. Those details are exactly what turn a generic “AI chat overlay” into an actual usable system.
In short, voitta-bookmarklet is interesting because it is not just a UI experiment. It is a compact architecture for attaching model-driven, tool-using assistance to arbitrary web pages while keeping execution local and leaving room for more integrations over time.
Voitta AI just released voitta-yolt, and it’s aimed at a very real problem: how do you let an agent move fast in the shell without giving it a blank check?
YOLO — You Only Live Once — is the vibe-coder’s operating principle: ship now, deal with consequences later.
YOLT — You Only Live Twice — is the correction.
No, it’s not a replacement for the auto mode; it’s a more fine-grained discerment: it gives Claude Code a second look before a Bash command (or the commands it invokes, which include actual code — e.g., Python, SQL) actually runs.
The problem it solves
Claude Code’s built-in permission system has an awkward gap.
Some commands are obviously safe, but still annoying to approve over and over. Others are wrapped in ways that make broad allowlisting dangerous.
Two cases matter most:
Arbitrary-execution wrappers.python3, bash, node, gh api, curl, kubectl, and friends are too powerful to wildcard-allow safely.
Compound shell commands. Loops, subshells, command substitutions, and bash -c '...' forms hide the actual inner commands from the simple outer matcher.
That means you either:
1. approve too much and weaken the safety model, or 2. approve everything manually and hate your life.
YOLT exists to get out of that false choice.
What YOLT actually does
YOLT installs as a Claude Code PreToolUse hook on the Bash tool.
When Claude is about to run a shell command, YOLT parses the invocation, walks the structure of the command, and classifies what it finds:
safe → auto-allow
unsafe → ask for review, with a reason
unknown → fall back to Claude Code’s default prompt
The interesting part is that it no longer treats the shell as a flat string.
The current release parses Bash with tree-sitter-bash, reconstructs argv from the AST, and then classifies each command node against rules in rules/shell.json. If the shell invocation contains inline Python, it delegates that body to a Python AST analyzer.
And it now covers a genuinely useful extra case: common SQL CLIs. sqlite3, psql, mysql, mariadb, and duckdb get their query text inspected so read-only commands like SELECT, SHOW, and .tables can pass quietly while mutating statements like INSERT, DELETE, DROP, .import, or .load get surfaced for review.
So this is not just “grep for scary words.” It’s structured analysis.
Why that matters
This is the real improvement over naive allowlists.
A normal matcher sees the wrapper:
bash -c "..."
for ...; do ...; done
$(...)
<(...)
YOLT walks inside those forms.
That means a loop full of read-only AWS inspection commands can be auto-approved, while a destructive operation buried inside a process substitution still gets surfaced for review.
That’s the right shape of safety tooling for agentic coding: less theater, more actual inspection.
The architectural shift
The sharpest detail in the release is that YOLT has already outgrown its first framing.
What began as a Python-script safety hook is now a more general shell-execution analyzer with language-specific followers.
hooks/yolt_analyzer.py — Python AST analysis when Python appears inline
That’s a better architecture than a pile of string heuristics, and the repo history shows exactly why the rewrite happened: quote-state edge cases, heredocs, substitutions, continuations, and shell grammar weirdness are not bugs you “finish.” They are why parsers exist.
Using a real grammar here is the grown-up move.
Practical wins
A few details make this more than a neat demo:
It supports both plugin install and manual hook install.
It explicitly warns that broad static allow rules like Bash(python3:) or Bash(aws:) can bypass the hook entirely.
It can use the user’s existing permissions.allow patterns as a secondary upgrade pass for otherwise-unknown inner commands.
The new SQL CLI handling is exactly the sort of practical expansion I like: not theoretical safety, but fewer prompts for read-only database inspection without waving through destructive schema/data changes.
It now defaults logging to ~/.claude/yolt.log, which makes dogfooding and debugging much easier.
And most importantly, the dogfood loop appears real. One recent pass through transcript history reportedly cut the classifier’s unknown rate from 60.2% to 11.7% by fixing a handful of recurring gaps. That’s the number I care about most, because it shows the project is being tuned against actual usage rather than imagined usage.
Why I think this matters
The broader point is not “Claude Code needs more hooks.”
It’s that agent safety gets much better when you stop treating the shell as an indivisible permission blob.
What you really want is a front-line gate for command execution: let the obviously safe paths go through quietly, and save human interruption for the suspicious stuff. That won’t replace every approval surface in an agent stack, but it can take a huge bite out of routine approval fatigue.
There is a big difference between:
aws ec2 describe-instances
aws ec2 terminate-instances ...
for svc in $(aws ecs list-services ...); do aws ecs describe-services ...; done
bash -c 'curl ... | sh'
A permission system that collapses all of those into “it’s Bash” is too coarse to be pleasant and too coarse to be trustworthy.
YOLT narrows that gap.
And the cleaner operational pattern is to pair that with direct API usage wherever possible. If a service already gives you a token to create a draft, update a post, or mutate a record, that is usually a better path than driving a browser through the same workflow just to satisfy the UI.
The real thesis
What’s new here is not just another safety wrapper.
What’s new is the move from tool-level permissions to structure-aware command understanding.
That is where a lot of agent tooling is headed, because the old model breaks down as soon as agents start composing commands instead of issuing one-liners.
If you want agents to operate with less friction without quietly turning root access into a vibes-based exercise, this is the kind of infrastructure you need.
Try it
YOLT is open source under AGPL v3 and available here:
And if you already installed it manually, the repo documents how to migrate cleanly to the plugin model.
That part matters too. Safety tooling people won’t keep updated is safety tooling that quietly dies.
Related: earlier we wrote about llm-tldr vs voitta-rag. YOLT sits in a different layer of the stack, but it comes out of the same practical question: if you are going to work with agents seriously, where do you put the guardrails so they help instead of getting in the way?
We went back and scraped all 36,915 forks (the number grew while we were counting). Every single one. Plus 9,423 pull requests. Three graphs, no black holes, no excuses.
Graph 1: The hockey stick that wasn’t quite a hockey stick
36,915 total forks. Peak: 3,402 on January 27. Average: 499/day.
The first fork appeared November 26, 2025. For nearly two months: nothing. A handful of early adopters per day, the kind of people who read Hacker News at 2am and clone things “to look at later.”
Then something happened around January 20.
Daily forks went from ~50 to over 1,000 in three days. By January 27, it hit 3,402 in a single day. That’s one fork every 25 seconds, sustained for 24 hours.
But here’s what the full data shows that the sample didn’t: it’s already declining. The peak was January 27. By mid-February, we’re down to about 1,000/day — still enormous, but the exponential phase lasted exactly one week. What we’re in now is the long tail. The viral moment came, the viral moment is going.
The cumulative curve tells the same story: a flat line, a vertical cliff, and then an inflection into deceleration. Classic viral adoption. The question isn’t whether it will keep growing — it will. The question is whether it levels off at 40,000 or 400,000.
Graph 2: Who actually builds anything?
7,591 of 36,915 forks (20.6%) have new commits. Threshold: code pushed more than 1 hour after forking.
This is the graph that matters.
In the early days — November, December — the commit rate was absurd. 60-90% of forks showed real work. These were people who forked because they intended to build. Small community, high signal.
Then came January’s tidal wave, and the ratio cratered. At peak volume, only about 10-20% of forks have any commits at all. The rest are what they’ve always been: GitHub bookmarks. One click, zero intention.
But zoom out from percentages and look at absolute numbers: even at 10%, that’s 300-500 people per day writing actual code on top of OpenClaw. The most recent week shows roughly 1,200 committed forks out of about 5,500 new ones. That’s a healthy project by any measure. It’s just a healthy project buried under 80% noise.
The trend line tells you something about open-source psychology: the harder a project is to use, the higher its commit rate. When OpenClaw was obscure, only competent developers found it. Now that it’s famous, everybody forks it and almost nobody builds anything. Same pattern as every framework that hits the front page of Reddit.
Graph 3: Who gives back?
9,009 fork PRs from 3,674 unique authors. 9.95% of forks ever sent a PR upstream.
One in ten. That’s actually remarkable for open source.
For context: most popular GitHub projects see PR rates of 1-2% of their fork base. React, with its 10:1 star-to-fork ratio, gets far fewer contributors relative to its fork count. OpenClaw’s 10% is unusually high — partly because the project is young and actively soliciting contributions, partly because the architecture (plugins, extensions, MCPs) makes it easy to contribute without touching core code.
The daily PR count has been climbing steadily: from single digits in December, to 50/day in mid-January, to a sustained 300-500/day now. Cumulative unique contributors crossed 3,500 and show no signs of flattening. Whatever is happening to the fork rate, the contribution rate is still accelerating.
That divergence — declining forks, accelerating PRs — is the best signal in this entire dataset. It means the project is transitioning from “thing people try” to “thing people commit to.”
What we got wrong in Part 1
Our original sample of the 100 newest forks found 19% activity. The full dataset says 20.6%. We were within a rounding error, which is either a testament to sampling theory or dumb luck. Probably both.
What the sample couldn’t show was the shape of the curve — the early period of 60-90% engagement that collapsed as volume exploded. The 20% number is real, but it’s an average across two very different populations: serious developers who forked early, and a much larger wave of tourists who forked because it was trending.
We also estimated “~2,400 forks/day” based on a snapshot. The real peak was 3,402. And by now it’s fallen to about 1,000. The snapshot caught a number that was already past its peak but hadn’t decayed enough to notice.
The numbers that matter
Forget 36,915 forks. Here’s what actually counts:
7,591 forks with real commits — people building things
3,674 unique PR authors — people giving back
~500 PRs/day at current pace — and growing
That’s not a fork explosion. That’s a contributor ecosystem forming in real time. The other 29,324 forks are scenery.
We’ll explain shoelace eventually. Promise.
Full dataset: 36,915 forks and 9,423 PRs scraped from the GitHub REST API v3 on February 17, 2026. All forks paginated (no sampling). Commit activity measured by comparing pushed_at to created_at with a 1-hour threshold to filter initial fork sync. PR data from GitHub’s search API.
These two facts are related in ways that are worth pulling apart.
What 34,600 forks actually looks like
A GitHub fork costs nothing — one click, two seconds. It’s a bookmark with delusions of contribution. So I pulled the data from GitHub’s API to see what’s actually going on underneath the vanity number.
GitHub’s API for listing forks returns a maximum of 400 results per request. You can sort by oldest or newest, so you get the first 400 forks ever created and the 400 most recent ones. The ~33,000 forks in between? Invisible. GitHub literally won’t show them to you. You’d need to scrape each fork individually or use their BigQuery dataset to see the full picture. I didn’t — so this analysis covers the bookends with a black hole in the middle. I’m not going to dress it up.
The growth curve
The first fork appeared November 26, 2025 — two days after the repo went public. For the next month: a trickle. One, two, three forks per day. Early adopters kicking the tires.
Then Christmas happened.
December 25: 10 forks. A 10x jump. People unwrapped laptops and had free time. The holiday week held steady at 5-10 per day.
January 1: 23 forks. Another 3x. By January 6, it peaked at 51 forks/day in the sample. New Year’s resolution energy: “this is the year I set up my own AI agent.”
And right now? ~100 forks per hour. 345 forks appeared in a 4.3-hour window. That’s a ~2,400/day pace.
The trajectory: 1/day → 10/day → 50/day → 100/hour.
Somewhere between people opening Christmas presents and Valentine’s Day, OpenClaw went from “interesting open-source tool” to “phenomenon.” Which is a convenient time for the phenomenon’s creator to get hired by the company that didn’t make it.
The 81% question
Here’s the part nobody talks about.
Of the 100 most recent forks — all created within the last hour of my sample — how many show any commit activity after forking?
19%.
The other 81% are untouched clones. Fork and forget. GitHub stars with extra steps.
But before you dismiss it: 19% of 100 forks per hour is still ~20 people per hour actually building something. That’s ~480 developers per day doing real work on top of OpenClaw. Not nothing. Especially for a project that, until yesterday, was one developer’s playground.
The ones who renamed their fork (and are apparently walking away from Omelas)
The most interesting signal isn’t volume — it’s intent. When someone renames their fork, they’re not cloning; they’re starting something new.
Highlights:
cl-core-mit-snapshot — someone freezing the codebase under MIT. Defensive forking. Just in case.
openclaw-x402-router — x402 payment protocol integration. Somebody’s building monetized agent infrastructure before the foundation even has bylaws.
reallyopenopenclaw — a philosophical statement in repo form. Already preemptively arguing with the future.
These are the 2% who forked with purpose. Watch them.
People aren’t just watching
OpenClaw’s stars-to-forks ratio is 5.7:1 (197K stars to 34.6K forks). For context:
React: ~10:1
Next.js: ~16:1
A low ratio means people are grabbing the code, not just bookmarking it. OpenClaw’s is unusually low. Whether that’s because the tool rewards customization, because the ecosystem hasn’t consolidated around plugins yet, or because people want to run it privately and not tell anyone — probably all three.
And now that the creator is inside OpenAI and the project is headed for a foundation? That cl-core-mit-snapshot fork starts looking less paranoid and more prescient.
The timing
Peter Steinberger announced yesterday that he’s joining OpenAI. Sam Altman said on X that OpenClaw will “live in a foundation as an open source project that OpenAI will continue to support.”
So let me get this straight: The tool was originally called ClawdBot — you can guess which model it was built for. The tool’s creator just joined OpenAI. The tool will live in a foundation that OpenAI sponsors. And 34,600 people have already forked the code, 81% of whom will never touch it again.
If you’re keeping score at home, a developer built a personal agent, originally called it ClawdBot (no points for guessing the model), made it go viral, got hired by OpenAI, and the project is now an “independent foundation” that OpenAI “supports.” This is like a Ford engineer building the best car on the market using Toyota engines, then getting hired by GM to “drive the next generation of personal vehicles.”
The claw is the law, apparently. Just not any particular company’s law.
What I couldn’t measure
Two of my three original questions remain unanswered:
✅ Fork creation over time — covered, with the API gap caveat
❌ Forks with independent commits — sampled 100, can’t do all 34,600 without days of API scraping
❌ Forks that sent PRs back to main — same problem, worse
A more rigorous analysis would use GitHub’s BigQuery dataset. This was a 30-minute investigation, not a thesis. But the 30 minutes told a story.
The real question
34,600 forks sounds massive. It is massive. But the real number is somewhere between 6,500 (19% active) and 700 (2% with intent). Still impressive, and still accelerating.
The open-source AI agent space is in its “everybody forks, nobody contributes back” phase. That’s fine — it’s how platforms grow. The interesting question isn’t how many forks exist today. It’s how many of them will still have commits six months from now, when the foundation has governance, when OpenAI’s priorities inevitably diverge from the community’s, and when the next shiny thing comes along.
History suggests: about 2%. But those 2% will be the ones that matter.
Data pulled from the GitHub REST API v3 on February 15–16, 2026. Fork listing capped at 400 per sort direction; findings are based on sampled bookends, not the full dataset.
Or: How Google and Microsoft Walked Into a Bar and Reinvented the Web, Worse
Google and Microsoft just co-authored a web spec together. Let that sink in.
The last time these two agreed on anything technical, IE6 was busy eating Netscape alive and “web standards” was an oxymoron. Now they’re back — holding hands under a W3C community group banner, gazing into each other’s eyes across a conference table, and delivering unto us WebMCP — a “proposed web standard” that lets websites expose “structured tools” to AI agents.
I have some thoughts.
What WebMCP Actually Is
WebMCP adds a new browser API — navigator.modelContext — that lets a web page register “tools” for AI agents to call. Each tool has a name, a description, a JSON Schema for inputs, and a handler function. Instead of AI agents scraping your DOM and squinting at screenshots like a drunk trying to read a menu, your website just… tells them what’s available.
Two flavors:
Declarative: You annotate HTML forms so agents can submit them directly.
Imperative: You write JavaScript handlers that agents invoke with structured inputs.
The Chrome team is very excited. They’ve published a blog post, opened an early preview program, and shipped it behind a flag in Chrome 146. VentureBeat wrote it up. Everyone is talking about the agentic web. The hype cycle spins.
The Problem WebMCP Solves
AI agents interact with websites by scraping the DOM, interpreting screenshots, and simulating clicks. This is fragile. It breaks when the UI changes. It’s slow and token-expensive (2,000+ tokens per screenshot vs. 20-100 tokens for a structured call). Every CSS class rename is a potential catastrophe.
This is a real problem. I’m not going to pretend it isn’t.
But here’s the thing: it’s a problem the industry created by ignoring the architecture that already solved it.
The Architecture That Already Solved It (You Didn’t Read It Either)
In the year 2000, Roy Fielding published his PhD dissertation describing the architecture of the World Wide Web. He called it REST — Representational State Transfer. You’ve heard of it. You’ve put it on your resume. You almost certainly haven’t read it.
(Don’t feel bad. Nobody has. That’s the whole problem.)
REST has one crucial, defining idea: HATEOAS — Hypermedia As The Engine Of Application State. Terrible acronym. Sounds like a sneeze. But the idea is simple and beautiful: the server’s response tells you everything you need to know about what you can do next. The links are in the response. The forms are in the response. The available actions are self-describing.
An HTML page already IS a “tool contract.” A <form> already IS a structured tool with defined inputs. A <a href> already IS a discoverable action. The entire web was designed from the ground up so that a client — any client, human or machine — could interact with a server without prior knowledge of its API, simply by following the hypermedia controls in the response.
“The HTML response is entirely self-describing. A proper hypermedia client that receives this response does not know what a bank account is, what a balance is, etc. It simply knows how to render a hypermedia, HTML.”
The web already had machine-readable, self-describing, discoverable interactions. It’s called… the web. Somewhere, Roy Fielding is thinking murderous thoughts.
So What Happened?
The industry collectively decided that REST meant “JSON over HTTP with nice-looking URLs.” Which is approximately as accurate as saying democracy means “everyone gets a vote on what to have for lunch.”
Fielding himself, in a now-famous 2008 blog post, tried to set the record straight with the restraint of a man watching his house burn down:
“I am getting frustrated by the number of people calling any HTTP-based interface a REST API… That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating.”
Reader, the industry did not listen. What followed was a twenty-year sprint in the wrong direction. We abandoned hypermedia for JSON blobs. We replaced self-describing responses with Swagger docs and API versioning. We built increasingly elaborate tooling — API gateways, SDK generators, GraphQL, tRPC — to paper over the problems caused by ignoring the one constraint that made the whole thing work.
And now, in 2026, having thoroughly ignored the architecture of the web while building on the web, we’ve arrived at the logical endpoint: a new browser API so that AI agents can interact with websites in the structured way that websites were already designed to support.
Roy Fielding is no longer thinking murderous thoughts. He’s past that. He’s watching the final scene of Chinatown. “Forget it, Roy. It’s the agentic web.”
The Declarative API Is Just Forms
This is the part where I need you to really focus. From the WebMCP spec:
“Declarative API: Perform standard actions that can be defined directly in HTML forms.”
They. Reinvented. Forms.
Google and Microsoft engineers got together — presumably with catering, perhaps even a whiteboard budget — and produced a specification to make HTML forms work for AI agents. HTML forms. The things that have been telling machines “here is an action, here are the inputs, here is where to send it” since 1993.
The <form> element is literally a structured tool declaration with a name (action), a method (GET/POST), and typed inputs (<input type="text" name="destination" required>). It has been machine-readable for thirty-three years. It is older than some of the engineers who wrote this spec.
But sure. Let’s add an attribute. Innovation.
The Imperative API Is Just RPC (Again)
The other half of WebMCP is the “imperative API,” where you register JavaScript handler functions that agents call with JSON inputs.
This is RPC. Specifically, it’s RPC mediated by the browser, authenticated by the user’s session, and invoked by an AI agent instead of a human. Which is a perfectly fine idea! RPC is useful. It has always been useful. SOAP did this in 1999. CORBA did it before that. Every SPA with a JavaScript API layer does it today.
The new part is navigator.modelContext.registerTool() instead of window.myApp.doThing(). The innovation is… a namespace. Alert the press.
The Security Section Reads Like a Horror Novel
WebMCP’s own specification describes something it calls the “lethal trifecta”: an agent reads your email (private data), encounters a phishing message (untrusted content), and calls a tool to forward that data somewhere (external communication). Each step is legitimate individually. Together, they’re an exfiltration chain.
The spec’s own analysis of this scenario? “Mitigations exist. They reduce risk. They don’t eliminate it. Nobody has a complete answer here yet.”
Nobody has a complete answer yet. They shipped it behind a flag in Chrome 146 anyway. This is the “we’ll add seat belts in v2” school of automotive engineering.
The destructiveHint annotation — the mechanism for flagging “this tool can delete your data” — is marked as advisory, not enforced. The spec literally says the browser or agent can ignore it. It’s a polite suggestion. A Post-it note on the nuclear button that says “maybe don’t?”
And there’s no tool discovery without visiting the page. Agents can’t know what tools Gmail offers without opening Gmail first. The spec proposes future work on a .well-known/webmcp manifest. You mean like robots.txt? Or /.well-known/openid-configuration? Or the dozens of other discovery mechanisms the web already has? Groundbreaking.
The Real Game
Now let’s talk about what this actually is, under the hood.
Google and Microsoft don’t control the API layer. They can’t dictate how backends expose services. But they do control the browser. WebMCP puts the browser — Chrome and Edge, i.e., Chromium with two different logos — at the center of every agent-to-website interaction.
Every AI agent that wants to use WebMCP must go through the browser. The browser mediates authentication, permissions, consent. The browser becomes the gatekeeper. If you control the browser, you control the chokepoint.
This is the same play Google made with AMP: take a real problem (slow mobile pages), create a solution that requires routing through Google’s infrastructure, W3C-wash it, and call it open. WebMCP takes a real problem (agents can’t interact with websites reliably) and creates a solution that routes through Chromium.
MCP (Anthropic’s protocol) connects agents to backend services directly — no browser needed. WebMCP says: no no, come through our browser. That’s not interoperability. That’s a tollbooth with a standards document.
What Should Have Happened
If we actually wanted AI agents to interact with websites reliably, we could:
Build better hypermedia clients. Teach AI agents to understand HTML — forms, links, semantic structure. The web is already machine-readable. We just need clients that aren’t illiterate.
Use existing standards. Schema.org, Microdata, RDFa, JSON-LD — mature standards for machine-readable web content. Google built an entire search empire on them. They work today.
Write APIs. If you want structured machine-to-machine interaction, build an API. REST (actual REST), GraphQL, gRPC — pick your poison. No new browser API required.
Use MCP where appropriate. For backend service integration, MCP does the job without inserting a browser into the loop.
None of these require a new browser API. None of them route through Chromium. None of them require Google and Microsoft to co-author anything.
The Cycle
This is the software industry’s most reliable pattern:
A good architecture is proposed (REST, 2000)
The industry ignores the hard parts (HATEOAS, hypermedia)
The easy parts get cargo-culted (“REST means JSON + HTTP verbs”)
Problems emerge from ignoring the architecture
A new spec is proposed to solve those problems
The new spec doesn’t mention the old architecture
Go to 1
WebMCP is step 5. The Chrome blog post doesn’t mention REST. Doesn’t mention HATEOAS. Doesn’t mention hypermedia. It talks about “the agentic web” as if machine-readable web interactions are a bold new idea that needed inventing in 2026.
Roy Fielding wrote the answer to this problem in his dissertation. In 2000. It’s free to read. It’s shorter than the WebMCP spec. And unlike WebMCP, it doesn’t require Chrome 146.
But sure. Let’s add navigator.modelContext. What’s one more API between friends?
