The editor accused the critic of hallucinating. It wasn’t.
In run mrs3ooy6, the critic flagged three passages in draft-1 as commodity content — writing a model could produce for anyone typing the same prompt. All three flags were correct. The editor reviewed the critique against the wrong draft, found nothing, and called the critic’s catch a hallucination. It wasn’t.
The three passages were real: a Bali-villa example at line 44, the phrase “read as entirely plausible in review” at line 64, and a bare 76% figure at line 97. The revision cut all three, producing draft-2, which published. The editor was handed the critique of draft-1 next to the text of draft-2 — not next to draft-1 — searched for the three passages, didn’t find them, and issued a verdict of publish anyway, on an article whose premise is that an unverified claim stated with confidence is exactly the failure this pipeline exists to catch.
The fix is one line of intent, committed to scripts/engine/pipeline.mjs: the editor now sees a critique of the draft it is actually judging. The pipeline re-runs the critic on the revised draft before the editor’s verdict, instead of carrying forward a critique of a draft that no longer exists. Separation of powers — a role that writes, a different role that can veto — is necessary. This run is the evidence that it isn’t sufficient on its own: the wiring between roles can be wrong in a way that produces exactly the false confidence the roles were built to prevent.
Why does separation of powers matter here?
Two roles in this pipeline can kill an article before it publishes: the editor-in-chief and the fact checker. The other five — writer, critic, SEO editor, art director, distributor — can only revise a draft, never stop it.
In run mrs3aw6f, the editor used that power on an article about a fact-checking registry that shipped with an inconsistency in its own numbers:
“The census arithmetic doesn’t reconcile — the article about a system built to catch exactly this kind of unverified inconsistency ships with an inconsistency in its own numbers.” — editor-in-chief, run
mrs3aw6f
In run mrs3iijt, the fact checker caught something worse: two citation-share figures with no source behind either, in a draft arguing that unverified numbers are the problem. One of them was a claim that Instagram allows 25 API-published posts a day. That number appears on no Meta page. It’s blog consensus, and it had been copied into our own registry by the research pass that was supposed to be debunking exactly this kind of number.
Meta’s own documentation, fetched 2026-07-20, gives two different figures in two different sections — 100 under Rate Limit, 50 under Limitations — and neither matches 25. We don’t know which of the two Meta figures is current for a given account type. We print both, with the source, and withhold the rest until someone runs the actual limit-check endpoint against an account we operate. That’s the fix that shipped: the pipeline now withholds the numbers of any conflict entry with no checkable source, rather than picking the more confident-sounding one.
2data/runs/2026-07-19-en-mrs3aw6f, -mrs3iijt, -mrs3ooy6 (committed) is the count for the first article this pipeline ever published: sent back once by the editor, once by the fact checker, before a verdict of publish.
The critic runs on a different model family. That’s not a stylistic choice.
The writer is anthropic/claude-sonnet-5. The critic is google/gemini-3.1-pro-preview. That pairing is the mechanism, not a preference: a model reviewing its own output shares its own blind spots, and rates its own ambiguous phrasing as clear because it wrote the phrasing.
Can a source verifier that fetches model-chosen URLs be trusted with API keys?
Not by default. A process that fetches URLs a model picked, while holding six services’ credentials, is a textbook SSRF primitive — the vulnerability and the feature are the same code path. On a default cloud runner that can reach 169.254.169.254, that endpoint hands out credentials over plain HTTP to whoever asks.
The fetch policy in scripts/engine/safe-fetch.mjs does all three. The test written against it found 2tests/safe-fetch.mjs; scripts/engine/safe-fetch.mjs (committed) — two holes the policy’s own reasoning had missed. A bracketed IPv6 loopback address ([::1]) was refused, but only because DNS happened to fail resolving it — not because the policy recognized it as loopback. An IPv4-mapped address that resolves to the cloud metadata IP was normalized by URL parsing into a hex form the dotted-form check didn’t recognize, and would have passed. Both were luck the test converted into policy, not foresight. The test is what makes the difference between luck and policy checkable — which is the same argument this whole article is making about editorial verdicts.
What does nine model calls cost, and where does the money go?
One article through the full pipeline — brief, draft, fact-check, source-verify, critique, revision, fact-recheck, SEO pass, art direction, verdict, collapsing to 9data/runs/*/run.json (committed) calls when no second revision is needed — costs $0.32–0.35data/runs/*/run.json (committed) for an article of about 1,500 words, measured across the first three runs.
| Role | Share of spend |
|---|---|
| Writer + editor-in-chief (claude-sonnet-5) | 81% |
| Critic (gemini-3.1-pro-preview) | 16% |
| Fact checker, SEO, art (gemini-flash-lite) | 3% |
Three roles generate judgment and run on the expensive model: the writer drafts the sentences, the editor-in-chief writes the brief and the final verdict, and the critic argues from a different vantage. Together they are 97% of the spend. The cheap model does the checklist work — the fact checker extracting every claim, the SEO pass, the art direction — because that job is arithmetic on another role’s output rather than a fresh judgment call: does the fact-checker’s report show a clean pass, does the census reconcile, does the flagged list match the draft. The editor’s bug in this article sat in that first, expensive tier, and it was not a reasoning failure — it was being handed the wrong draft to reason about.
The fix from run mrs3ooy6 adds a call to that arithmetic: the critic now re-runs against the revised draft before the editor’s verdict, on every article that goes through a revision. That raises the per-article cost above the $0.32–0.35 range shown above for any article that trips a revision, since the table was measured across the first three runs, before this fix was committed. We haven’t re-measured cost with the rerun in place.
What doesn’t this fix?
None of this makes the writing good. The fact checker blocked two unsourced figures in mrs3iijt. The editor killed a census inconsistency in mrs3aw6f. Neither role added a fact the company didn’t already have — they only refused to ship without one.
A value review of the corpus scored this engine’s first published article at 80 out of 100, against a legacy corpus of AI-written articles scoring 30 to 40 on the same rubric. The gap wasn’t better prose — the model producing the sentences is the same kind of model either way. The gap was that the engine’s article carried something a prompt alone couldn’t produce: a real run log, a real cost figure, a real security test result. This exact article is the second attempt at proving that point. The first draft of it scored in the 30-to-40 range, on the same rubric, for the same reason every commodity article does: it described the architecture instead of showing it fail and get fixed. The fix wasn’t a better prompt. It was going back to the logs.