A LinkedIn post called a fifteen-minute Boris Cherny video mandatory viewing and listed the stack to copy: plan mode, CLAUDE.md, subagents, hooks, worktrees. We had already built every item on it. In the talk itself, Boris says “maybe don’t listen to the LinkedIn influencers“, and his actual advice is the opposite of a checklist: “every six months, delete your [CLAUDE.md], delete your skills, delete your hooks.” So we measured what ours was worth before deleting it.
What our setup bought
Three arms, same model (claude-sonnet-5), same tasks: our full setup; claude --bare, which strips nearly everything; and --safe-mode, which keeps Anthropic’s default prompt and drops only ours. Our setup added about 28,800 tokens to every request: 53,300 against the default’s 24,500.
On five code questions, scored out of 12 by a judge that resolves every citation, three runs each:
| Setup | Mean score | Cost per answer | Fabricated citations, 15 answers |
|---|---|---|---|
| Ours, full | 11.00 | $0.33 | 6 |
| Anthropic default | 10.87 | $0.19 | 13 |
| Bare | 10.47 | $0.19 | 23 |
Ours and the default are the same within noise; the 95% interval on the difference runs from −0.7 to +1.0. Ours costs 1.7 times as much. On six replays of real pull requests, every arm passed the repository’s three checks in all 18 runs, and ours cost $0.97 a run against $0.61. So: neutral on quality, 1.6 to 1.7 times on cost.
And one hook did worse than nothing. Our command-safety hook blocked the repository’s own validation script in every one of our replay runs, 23 refusals across six runs, so the agent “verified by hand”. That is the false positive part 4 fixed.
The bare arm is the useful contrast. It cost the same as the default and fabricated nearly twice as many citations. Anthropic’s prompt earns its size; ours, on these tasks, did not.
What we deleted, and what we kept
We merged two global instruction files into one and cut it from 18.7 KB to 3.2 KB: dead pointers, rules the harness already enforces, style mandates. What stayed is what an experiment defended:
- the independent reviewer, a model from another provider, which found defects in every diff across our experiments while the agent’s own tests passed (part 1);
- the rule that the agent does not write tests unless asked, kept on that same evidence;
- the credential rules;
- task, guardrails and done-when, instead of step lists (part 3).
Not everything went. The hook that injects a “mandatory skill evaluation” into every prompt still fires; it fired on every message while this post was written. Deleting is a practice, not an event.
Someone else’s checklist
The same arithmetic applies to the popular bundles. mattpocock/skills is a good one, with 38 skills and some 292,000 stars, and adopting it whole would be exactly the packing we just measured. Our rule now is to bring in a skill only against a stumble we have seen, never as a bundle.
Read skill by skill, it agrees with us more than it disagrees. Its test-driven-development skill names the “tautological” test and says “expected values must come from an independent source of truth”, which is part 1’s conclusion in one line. Its code review runs two reviewers in parallel sub-agents of the same model. Separate context is not an independent model; we would add it to cross-provider review, not swap one for the other. We have not tested same-model review, so that is a position, not a finding. Its spec skill asks for “a LONG, numbered list of user stories”, which leans against what part 3 found: length did not help, saying what done looks like did.
What a star counts
About those stars: they are not a scoreboard, and ours are in single digits. A GitHub star is one click, by a logged-in person, on the repository’s page. It costs nothing, requires no use, and is almost never taken back. It counts people who reached the page and felt like clicking, which mostly measures the author’s audience and who linked to it. Borrowed eyeballs open the door.
Our skills are installed through the plugin marketplace, which fetches the repository without anyone visiting the page where stars are given. The install path we built bypasses the place the metric is collected. The number that looks like usage instead, git clones, counts machines: plugin updates, our own sessions, CI. Pulls, not people.
That is the whole series in one move. Before reading a number, say what produced it.
Lightly held
Strong opinions, weakly held, or in our case opinionated but lightly held. What would change our minds: a task type where the full setup beats the default beyond noise, or a model generation where the bare arm stops fabricating. We will rerun the ablation when the next model ships, not on a calendar.
Limits: three to five runs per cell, claude-sonnet-5, a handful of task types, two repositories. The cost ratio is solid; “neutral on quality” means we could not see a difference, not that there is none.
Part 5, the last of a series on what our process experiments found. Part 1: the agent’s tests agreed with the agent. Part 2: our eval read the answer key. Part 3: say what done looks like. Part 4: widen a filter by layering, never by editing.
