In my system, a multi-year analysis could legitimately mean at least three different things (tracking the same students’ growth across years, comparing this year’s cohort against last year’s, or trending school-wide results over time). So I defined the three ways we do it and wrote selection rules based on the shape of the district’s data. That handles the ambiguity machines can settle.

For the ambiguity they can’t: when someone asks the system to build a district’s renewal report and that district runs our courses in several subjects (e.g. English and math), the system stops and asks me which subject I wanted instead of guessing.

Ambiguity resolves through a defined menu of interpretations with selection rules — and when the rules can’t decide, the system asks the operator instead of guessing.

Enumerate the legitimate interpretations before the model ever meets them. Encode the rules for choosing. Gate whatever remains on a human. The model generates within a chosen interpretation — it never chooses silently.

I audited one full, end-to-end run of my report-building engine. Across the run’s 164 logged events, the output drifted from the spec in six distinct ways — even though the three reference documents that define the correct behavior sat in the working directory the entire time, available to read. Separately, a 1,000-line skill spec that ran clean when I supervised it interactively started silently skipping steps the moment it ran as a background agent. Same spec. Same code. Same data.

Anything deterministic must be code, not prose the model re-interprets on every pass. The longer the run, the more this matters.

A validating helper the model calls with arguments — it never writes the fixed shape itself. The clearer your prose, the more tempting it is to trust it. Don’t. Better prose isn’t the answer; not-prose is.

In that same 164-event audit: sql_query is a valid subtype: in my schema, but on 39 of 164 events the model wrote it into the type: field instead, which is the parent field of subtype: and the wrong one to use here; an invented entry: START/END field appeared on 34; nine invented subtypes fell outside the schema’s allowed set of values; and the log file itself was written to the wrong directory.

The producer of a fixed shape validates structure and rejects unknowns. The model passes data — never format.

A small CLI owns the schema: it allocates IDs, stamps timestamps, hardcodes the path, validates every enum, rejects invented fields. Five of my six drift modes disappeared by construction — the model can’t drift a file it no longer writes.

While registering a new SQL query with verified logic, the model hand-typed the expected result of that query instead of running it. The typed value was 530x off from what the warehouse actually returned.

This one incident put a hard gate on expectation strings (e.g. “this query should return X”): computed, never authored.

No value reaches output unless execution produced it during this run. Result strings bind to the query that made them — never to memory.

Separate compute from narrate. Registering a metric forces the query to execute at registration time, so the recorded expectation is real by construction. The narrator may reference computed values by name; it can never mint one.

The fix: narrative templates carry tokens instead of numbers. A sentence is authored as “{funnel_enrolled} students enrolled in ChalkTalk,” where {funnel_enrolled} names a registered SQL query or Python calculation. At render time it becomes “12,651 students enrolled in ChalkTalk.”

Interpolated values render visibly distinct from authored prose — a reviewer sees at a glance which words are data, and can click a number to land on the query that produced it.

Every claim traces to a recorded event — and the trace is inspectable. Data-bound text and authored prose must be distinguishable at a glance.

One read model for narrative and evidence: numbers live as tokens bound at render time, never retyped. A claims file with source and reasoning per claim; a halt when a narrative number matches no recorded event. Provenance that exists but can’t be read is provenance that doesn’t exist.

I built a whole validation layer because of this zone. The checks: grain (each row unique on declared columns, catching fan-out), rollup (recomputing the same number from its finest-grained rows gives the same total — one query checked against its own math), scope (no rows from outside the declared district, curriculum, and year), source-consistency (the declared table matches what the SQL actually joins), and reconciliation (separate outputs checked against each other — the counts published in one section must sum to the headline in another, and independent sources must agree within tolerance).

Declare invariants on the output shape — uniqueness, parts-equal-whole, no leakage — and mechanically verify them at every step. Verify the middle, not just the ends.

The invariant rides as a declaration on the data model itself and is checked per-query at runtime. Severity is a product decision per constraint: fan-out and broken rollups halt; a possible scope leak proceeds but gets flagged where the human and the narrator both see it.

A report cover read: “reaching 334 of 440 enrolled students and a path to doubling the number of students with paired before-and-after evidence.” Every number was right — the doubling meant 109 going to roughly 220. But put “doubling” next to “334 of 440” and the reader’s eye computes an impossible 668. Correct data, false reading.

Check how a number will be read, not only whether it is right. A multiplier must bind explicitly to its base.

A render-time check for perception traps — mine detects a multiplier landing near an unrelated base and halts unless the sentence carries its own binding: “doubling (from 109 to ~220).” Data-correctness gates can’t catch this class. It needs its own gate.

My version: phase 2 computed exact numbers, but they reached phase 3 only as a lossy conversational summary; then a phase that did no real work poisoned everything downstream — all while the final report still read beautifully. Until each phase boundary became a checkable artifact, localizing a cascade meant re-reading a 90-minute transcript by hand.

Deep orchestration must decompose into phases that each prove their own correctness — or a coherent-sounding whole hides a broken part you cannot find.

Checkpoints between phases — downstream reads the file, never the prior prose. Per-phase execution counts. Reconciliation at phase boundaries, so a dead layer surfaces after phase 1 instead of after the run. Orchestration depth raises the stakes of per-phase proof; it never lowers them.

One run executed every phase inline instead of dispatching them: substantive-looking deliverables, zero telemetry, and about 115 false alarms from confused downstream checks. An earlier run had silently dropped an entire analysis phase and two customer PDFs — nothing checked for them, so nothing noticed. The hardening that followed cut detection from ~90 minutes to ~30 seconds.

Every mandated step proves it ran. A step that cannot show execution evidence halts the run. Zero evidence is a failure — never a clean run.

A completion contract (a machine-readable list of what each phase must produce) plus per-phase execution counts means a hollow phase halts before the next one builds on it. The elegant endgame: the retry path re-dispatches the phase, and a dispatched subagent physically cannot run inline — the architecture cures its own failure mode.

An empty environment variable made my logging hook silently do nothing. Hooks installed mid-session didn’t fire until restart: one district’s run captured 0 of 26 expected events and finished “successfully.” The report was right. The telemetry was fiction — and nothing knew.

An end-of-run assertion that the telemetry captured reality — and the checker shares code with the thing it checks, so the two can never drift apart.

Reconstruct what should have been recorded from an independent source — the session transcript — and diff it against what was recorded; hard-fail on zero-capture. A detail I’m proud of: the gate imports the recorder’s own decision function instead of re-implementing it. The anti-drift lesson, applied to the anti-drift tool.

A monitoring loop I launched watched for a status its target never publishes on that channel. The loop was healthy and the telemetry was honest — it would have polled to its timeout reporting a clean “still waiting,” forever. I caught it because the silence itself eventually read as wrong.

A loop’s termination condition is itself a checked artifact: proven able to fire against a known-done state before the loop is trusted with anything.

Test every stop condition once against a state where it must fire — the falsifiable-predicate discipline, applied to loops — and make timeout a fail-closed verdict that reports loudly, never a quiet exit. Its sibling is the Lying Recorder (C10), one axis over: C10 is a run whose record you can’t trust; this is a run that can’t hear it’s done.

When the system ran unattended, the steps it skipped were never the judgment calls — they were the boilerplate. Every run ends with the same four wrap-up commands. Each prose handoff was one more chance for attention to wander — exactly where the skips clustered. One 90-minute run wasted about 50 minutes on work it had abbreviated and then had to redo.

Code drives sequencing. The model is invoked only where the next step requires judgment.

A driver script picks the next step from state. It calls the model surgically — choosing the story angle, drafting narrative, talking to the operator — and runs helpers directly for everything else. Skipping becomes structurally impossible, and you stop paying inference prices for couriering.

I run all three shapes in one system, deliberately. Heavy data phases fan out — context isolation and parallelism. Judgment moments and operator gates stay inline, where the orchestrator can see and ask. Choosing wrong in either direction cost me: sequential drift on long runs, and contained subagents that couldn’t surface the question they should have asked.

Orchestration shape is a first-class design decision trading accuracy × performance × control — chosen per step, never globally.

Fan out for isolation and independent parallel work. Stay inline wherever a clarification gate or a human question lives. A driver mediates between the two. This is more than a telemetry choice — it decides what the system can notice and what it can ask.

My first skill was a 600-line monolithic file — every rule, every reference, loaded up front. By line 300 the agent had forgotten the constraints from line 50. It performed worse than having no skill at all.

Context loads progressively: start small, navigate to detail on demand, stop at the shallowest layer that answers the question.

Layered structure — a small entry point with dispatch logic, phase files that load when their phase fires, references that load when pointed to. The budget you’re actually managing is attention, not tokens.

Mid-run, phase 2 produced exact metrics — paired_students: 247. The conversation compacted between phases. Phase 3 read “about 250 paired students” from the summary and built the customer-facing artifacts on it. The report was off by single-digit percentages everywhere, for no visible reason.

Persist load-bearing facts to disk the moment they exist. Files survive compaction; conversation prose does not.

A checkpoint file written at every phase end, in structured YAML, so a value like paired_students: 247 is data, not a sentence a summarizer can soften into “about 250.” Narrative may be summarized; hard numbers are written to disk before they are ever presented. Downstream phases read the file, never the prior prose.

The same branch, path, and formatting mistakes recurred session after session until each became a written rule in a file the agent loads at startup. Then the sharper proof: I came back from my two-week honeymoon and couldn’t remember some of the finer details of how the “shared learnings & rules” system I had built four weeks earlier worked — which file fed which reviewer, where a lesson was supposed to live.

Every correction becomes a rule in a known home that the right reader loads automatically — at write-time and at review-time.

Route the lesson by who needs it: personal habits into per-user memory; repo standards into committed rule files mirrored into the AI reviewer’s config. One lesson, two readers — the writer never makes the mistake, and the reviewer catches it if it slips through anyway.

I shipped a redesign, self-reviewed the diff, and it looked clean. A different vendor’s model (Codex reviewing Opus’s work, in my case) then returned 9 findings — 7 were real bugs that would have gone to production. The disagreement between models was the signal.

A second, different model reads every change before it ships — and you validate its findings against source, because it is confident when wrong, too.

Cross-model review as a standing gate: one model writes, a different one reviews, a human adjudicates with the source open. Model diversity — different training, different priors — catches what repetition can’t.

My skills share a library of capabilities — telemetry, shared Python helpers, reference data files. Each skill declares which capabilities it adopts and which it opts out of, with a justification, so one canonical implementation serves many skills and a linter can check each declaration against reality.

The linter’s first version emitted an advisory note for every declared opt-out — which it never actually verified. Any skill could copy an opt-out clause from a neighbor, forget to update the wording, and pass forever. The fix made the check enforceable, with a test for exactly the copy-and-forget case.

Every declaration is machine-checked against reality. An unverified promise is treated as false.

The dial-and-machine pattern: a declaration you write (the dial) is only meaningful when automatic machinery reads it and fails when it doesn’t match reality (the machine). A dial with no machine is decoration.

I built a baseline-drift mechanism to catch numbers shifting between runs. Across weeks of real runs: one baseline ever committed, zero drift comparisons performed. I deprecated it — and deleting it is what forced the better mechanism (binding results to live execution) to exist at all.

Build the gate when a failure proves it’s needed. Delete dormant machinery — its presence implies a coverage that isn’t there.

Keep an explicit ledger: what’s automated, what’s still prose, what’s a promotion target. Promote on proven pain. Retire on proven disuse. The ledger of what you haven’t hardened is as load-bearing as the hardening.

Ten of my nineteen map onto it. The drift family (A1–A2) is their “disobey task specification.” The Silent Skip (C9) is their “no or incomplete verification” — 8.2% of every failure they annotated. The Compaction Cliff (D15) is their “loss of conversation history.” Single-Model Blindness (E17) lives in their “incorrect verification.” And the Deaf Loop (C11) is their “unaware of termination conditions” — I hit it the week this post went up, and it earned its card the way the rest did: by failure. Their taxonomy came from reading traces; this one came from being burned by runs. We met in the middle.

Four of their modes are missing here, and the reasons matter more than the count. Three — conversation reset, information withholding, ignored agent input — can only happen where agents talk to each other freely. My phases hand off through checkpoint files, never conversation, so there is no conversation to reset: those were engineered out before I knew their names. The fourth, step repetition, is bounded out — retries cap at one re-dispatch, and a completed phase leaves artifacts on disk, so a repeat announces itself.

Their largest failure category is system design — better architecture, not better prompting, is their prescription, and prompt-improvement interventions plateaued in their own experiments. That’s this post’s argument arrived at from the other side. And the working relationship between the two catalogs is the one PDR predicts: you don’t need the taxonomy in advance, because the failure names itself on a run. But it has a name — and checking your scars against the published map is how you find out which ones are structural.

Almost nothing overlaps directly, because their map was written for a different lens — agents under attack rather than agents malfunctioning. But the twin pairs are striking. Their memory poisoning is my Re-Learned Lesson (D16) with a villain: they fear memory gaining lies; I fear it losing truth. Their inter-agent trust escalation — an orchestrator trusting a sub-agent’s self-asserted claims without verifying — is the Polite Fiction (E18) weaponized. Their session context contamination is Context Dilution (D14), seeded deliberately instead of self-inflicted.

The gap is the threat model, and it is by design on both sides. Every mode on their map assumes an attacker, because charting hostile input is what a red team is for. Every mode on mine happened with no adversary anywhere, because I was building an internal reporting engine — the only thing that could hurt the system was the system itself. Both maps are real. They chart different oceans, and a production system sails on both.

Their June 2026 mitigations — context provenance tracking, structured separation of trusted from untrusted content, never trust a self-asserted claim — are this catalog’s fix shapes, rediscovered from the attack side. When the security people and the reliability people converge on the same architecture independently, that architecture is probably load-bearing.