I let an AI proficiency scanner read thirty days of my Claude Code transcripts. It handed back a 349 out of 1000 — a C — and, more interestingly, two flat zeros: Planning: 0. Customization: 0. Agent dispatches: 5.

I spend most of my time running multi-agent squads and writing skills. So either I’m worse at my own workflow than I thought, or the instrument is measuring the wrong thing. It turns out the source is right there — Elastic-licensed, readable — so I did the thing the score didn’t credit me for, and read it.

What it actually measures

The scanner is a transcript-shape heuristic. Volume it gets right: 168 sessions, 11.3k messages, 524M tokens — all there. The trouble is the three dimensions meant to capture sophistication, each pinned to a single tool-name signal:

• Orchestration is the literal tool named Agent — and “parallel agents” only counts when two or more Agent blocks fire in one assistant turn.
• Planning is the ExitPlanMode tool. That’s the whole definition.
• Customization is a Write or Edit whose path ends in SKILL.md / CLAUDE.md / .mcp.json.

Three proxies, each standing in for a competence, each satisfiable by clicking the corresponding button.

Why my real work scored zero

Regular readers know the shape of my orchestration skill: an architect derives the parallel set, then each issue gets a developer in its own git worktree, an adversarial reviewer, an SDET, a productivity engineer. That is the opposite of “two Agent calls in one turn.” The squads run as background agents and separate worktree sessions, each producing its own top-level transcript. From the scanner’s vantage, that isn’t orchestration — it’s a pile of independent user sessions. The one line that decides this, is_main = "subagents" not in path, doesn’t just miss cross-process orchestration; it actively penalizes it, because every coordinated sub-session inflates the denominator as a plain session.

Planning is worse, and funnier. My whole doctrine is start with a conversation, not a spawn — the hard, valuable artifact is the parallel set an architect reasons out after reading the issues. None of that touches ExitPlanMode. So a planning discipline more deliberate than the feature scores lower than one press of the Plan button. The map graded me on whether I visited a specific city, not on whether I arrived.

And Customization: I author SKILL.md files as routine work, but from worktree sub-sessions and through git and PRs — not via the Write tool under ~/.claude/skills/ in a session it happens to be watching. The scanner even records prs_opened; it just doesn’t spend it on the dimension that would have caught me.

The actual defect

This is a construct-validity failure, not a bug. Every proxy assumes competence lives inside one process boundary and expresses itself through a named UI feature. Mine lives across boundaries — worktrees, sibling sessions, gh — and in git history the instrument never opens. It measures feature adoption and calls it proficiency. Goodhart is right there: I could raise the number tomorrow by making my workflow worse — two Agent calls per turn, a ceremonial ExitPlanMode, a Write I don’t need — and the score would thank me for the regression.

What I’d change

Same thesis I keep landing on: the score is the spaceship; the missing instrument is telemetry that reads across boundaries. Concretely — correlate sibling worktree sessions plus gh pr create into a single orchestration event; score customization from committed SKILL.md diffs, not in-transcript writes; credit the gh issue view fan-out and the long analysis turn that precede a spawn as the planning they are. I’ve written it up as an issue against the plugin, and I’ll send the scanner half as a PR. The server-side scoring isn’t in the open repo, so I can hand them a better signal but not a better weight — which is its own small lesson about what “open source” buys you.

None of this makes the tool useless. A cheap proxy over free telemetry is a reasonable place to start; I’d have started there too. But a proxy has to know it’s a proxy, and this one reports a C with the confidence of a measurement. The gap between what it counted and what I did is, as usual, the only part with information in it.