Twice, the bug was the error message

An agent reads your error messages more carefully than any user ever has, and then acts on them. Give it a code and it guesses. Give it the cause and it reasons. Tell it what to do next and it tells the user. We learned this twice in one afternoon, on one page.

The page that said no

On 2026-11-13 I was helping a friend with a project built on Lovable. The question had nothing to do with Voitta. Partway through, mostly out of frustration, I clicked the Voitta bookmarklet on the project’s page.

The widget loaded. Chat worked. Every script the assistant tried to run on the page failed, even return 1+1, and the model told me it could confirm the page title and nothing else.

Error one: a code without a cause

What the model saw was {"ok": false, "error": "primitive_error", "message": "eval_threw"}. Our backend had the page’s full error in hand, message, stack and console logs, and reduced it to a code on the way to the model. Given “it threw”, the model could only give up.

Once browser_eval passed the whole error through, the model named the cause on its next try: the page’s Content-Security-Policy forbids eval, and retrying will not help. Same model, same page, same question. Only the error text changed.

The policy is script-src 'nonce-…' 'strict-dynamic' 'wasm-unsafe-eval' 'self' 'report-sample', with no 'unsafe-eval'. That is ordinary hardening, and the page was right to have it. The assumption was ours: every page tool we had ran through eval.

Reading a page needs no eval. read_page is a DOM read with a selector and a size cap. It was proposed, built and shipped the same afternoon, after 23 end-to-end checks against a local page served with the same policy.

Error two: a cause without a next step

On the first live try the model picked read_page on its own. The call failed, and the model told me the site was blocking that too: “a hard block from the site”.

It was not. The backend had been upgraded, but the tab still ran the widget it had loaded before the tool existed. That widget answered unknown browser tool read_page. Clicking the bookmarklet again would not have helped either, because the widget does not mount twice on one page. The error said none of this, so the model filled the gap with the most plausible story it had.

The fix is again a message, and this time it carries the next step: the site is not blocking anything; reload the page and click the bookmarklet again. In a later test turn the model relayed it as written: “The Voitta widget in your tab is outdated and doesn’t support reading the page. Please reload the page and click the Voitta bookmarklet again, then ask me to try again.” No blame on the site.

After a reload, the assistant read the whole page as text, about 8,400 characters. It was the first time it could read that page at all.

The rule

An error an agent will read has three parts: what happened, why, and what to do now. The first error had only the first part, so the model gave up. The second had the first two, so the model reasoned its way to a wrong answer. A wrong diagnosis said confidently to a user costs more than “I don’t know”, because the user acts on it.

What else real use found

The afternoon took seven pull requests and six releases (v0.2.16 to v0.2.21). Three of the faults appeared only because the tool was in real use rather than in a test:

  • A fresh clone would not build on Python 3.15: onnxruntime, which chromadb needs, has no wheel for it.
  • I pasted a screenshot into the chat with no text, and the turn failed with text blocks cannot have empty text. Images were supported; the message builder always added a text block, even an empty one.
  • Five releases in one afternoon meant five overlapping image builds, and an older one finished after a newer one and moved the :latest tag back to its own version. Now only the newest release moves :latest.

What the assistant still cannot do on a page like this is act on it. Clicking and typing go through eval. Asked where a setting was, it said it could not click around for me and gave me the steps instead. That was the right answer, given the rule above.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.