Supervisor and Subagents

By Yianna Kokalas & Claude

How this paper was written. This is a working paper from a production side project: software built for the Magic: The Gathering community, with real users, built and operated by one engineer (an enterprise engineer by day) together with Claude, Anthropic’s AI model. Every mechanism claim was fact-checked against the running code before publication. The audit found nine places where our own documentation had drifted, and two claims we had repeated for months were tested and killed by measurement before this went out; all of it is published in sections 7 through 9 rather than quietly fixed. Both names on the byline mean it.

Position: the working unit of serious agentic engineering is not one strong agent with a long leash. It is a supervisor delegating to role-scoped subagents, each with its own context window, its own permissions, and a model chosen for the risk of its specific job. The supervisor holds the plan and the verdicts; the subagents hold the work. This is the same trust arc the previous paper in this series walked (The Drain): autonomy is extended only as far as verification allows, and role boundaries are what make verification tractable. A reviewer that shares no context with the implementer cannot inherit its blind spots. A fix agent that cannot merge cannot bury its own mistake.

This paper is also an audit, and the audit question comes first:

Do role-scoped subagents on deliberately different models beat one strong agent, and is each of our model assignments still evidence-justified, or a habit nobody re-measured?

The honest answer, argued in section 7: the role-scoping is incident-justified and we would rebuild it; the model assignments mostly are not. Of 33 model-assignment slots in the system, 18 carry no recorded evidence at all, 4 are backed by real measurements (all of them the local models, not the frontier ones), and the rest are recorded decisions or incident-backed pins that were reasoned once and never measured. One flagship topology has direct measured counter-evidence against itself, and the skill now says so in its own body. We print all of it. The paper’s response to the unevidenced rows is not backfilled justification: section 7.1 restates each contested assignment as a falsifiable claim with a named kill condition and the live alternatives beside it. Two of those claims were tested before publication. One killed a number from the previous paper in this series. The other killed the system’s most-repeated security rationale at 0 refusals in 12, for thirteen dollars.

Every mechanism claim below was verified against the code and skill definitions at HEAD on 2026-08-30. The file:line citations throughout reference our private orchestrator code and skill definitions, so a reader cannot follow them; they are retained anyway, as evidence that each claim was checked against a specific line of running code rather than recalled from memory.

1. The shape of the unit

Some vocabulary, so the paper reads cold. The system is a planning vault: a git repository of markdown ticket specs, plus skills (versioned prompt programs a session can invoke) and a headless dispatcher called the drain that turns approved tickets into merged pull requests with no human in the loop. A “supervisor” here is the one session that owns a run: it reads the ticket, spawns every worker agent itself, scores their results, and is the only writer of consequential state. A “subagent” is a worker spawned for exactly one role, with a fresh context containing only what the role needs.

Three properties define the unit, and none of them is stylistic:

1. The supervisor spawns everything. The harness blocks subagents from spawning subagents, so delegation is one level deep by construction (implement/SKILL.md:11). This is a constraint we chose to lean into rather than fight: a flat topology means there is exactly one place where verdicts are aggregated and exactly one agent whose claims must be re-verified against ground truth.
2. Roles are scoped by permission, not by politeness. Investigator agents are read-only by convention and by tool restriction (agents/index.md:3). The implementation agent may not merge its own pull request or advance ticket state (implement/SKILL.md:218). The fix agent fixes; it does not re-litigate the review (“it is a fixer, not a critic”, implement/SKILL.md:281). When a role needs isolation from the supervisor’s own working tree, it builds a separate git worktree and works there.
3. Models are assigned per role, per risk. The strongest models go where judgment is scarcest (adversarial review), cheaper models go where the work is bounded (mechanical synthesis, batch classification), and one entire task family runs on local open-weight models where the economics demand it. Whether each assignment is still justified is the audit of section 6.

The trust-ladder frame carries over from the previous paper: none of this delegation was trusted on day one. Skills ran interactively with a human watching before any supervisor was allowed to run them over subagents, and subagent output is treated the way the drain treats dispatch output: as a claim, verified against ground truth (git log, GitHub PR state, the test suite) before anything advances. The specific rule “confirm via git, not via the agent’s say-so” was earned: named background subagents were observed going idle and never returning result blocks, so the flagship pipeline spawns its agents unnamed and synchronous and verifies their work products directly (implement/SKILL.md:203,275).

2. The flagship: the /implement pipeline

/implement executes one ticket end to end: implementation, two independent reviews, a bounded fix loop, then merge eligibility. It is the pipeline the drain dispatches headlessly, which means every role boundary in it must hold with nobody watching.

                          supervisor (thin dispatcher)
                          reads ticket, spawns all agents, scores verdicts,
                          only writer of ticket state
                               |
            +------------------+
            |
            v
  IMPLEMENTATION AGENT
  fresh context, builds its own
  git worktree, implements the
  ticket, runs the tests
  model: inherits the dispatcher
  may NOT merge or advance state
            |
            |  work product verified via git, not via self-report
            v
  +---------------------------+---------------------------+
  |                                                       |
  v                                                       v
  CRITICAL REVIEWER                          SECURITY REVIEWER
  correctness, design, tests                 auth, injection, secrets
  model: Fable interactively,                model: Opus in ALL modes,
  Opus under headless drain                  deliberately never Fable
  (cost); on refusal or death,               (false-refusal risk on the
  re-spawned on Opus; a missing              exact auth/JWT/credential
  review is NEVER scored PASS                code it must read)
  |                                                       |
  +---------------------------+---------------------------+
                              |
              both PASS?      |      findings?
                              v
                     FIX AGENT (model: sonnet)
                     applies review findings only;
                     merge-state gate runs first so a fix
                     never lands on an already-merged PR
                              |
                              v
                     BOTH REVIEWERS RE-RUN, FRESH CONTEXTS
                     (max 2 fix cycles, then verdicts decide)
                              |
                              v
                     merge eligibility: human opt-in flag AND
                     validation PASS AND tree-integrity PASS
                     (missing = fail closed) AND both reviews PASS
                              |
                              v
                     gh pr merge --auto --squash
                     (the pipeline never writes "done";
                     the drain does, after deploy verification)

The texture is in the gates, and each gate has a scar:

• Fresh re-reviews. After a fix cycle, both reviewers re-run with fresh contexts rather than being asked “is it fixed now?” (implement/SKILL.md:401). A reviewer that carries the prior conversation has already committed to a framing; a fresh one has not.
• The merge-state gate. Before every fix cycle the supervisor checks actual PR state, because in June 2026 a fix was committed to a branch whose PR had already merged (implement/SKILL.md:339-348). Findings against a merged PR become a follow-up ticket, never a commit.
• PASS-with-warnings is PASS. An operator ruling dated 2026-08-19, made after a green PR sat unmerged for a day over advisory nits (implement/SKILL.md:402). Only an explicit FAIL blocks.
• A missing review is never a pass. If the critical reviewer’s model refuses or dies, the same prompt re-runs on Opus; the absence of a verdict is treated as the most suspicious outcome, not the cleanest (implement/SKILL.md:279).
• The self-collision ban. Once preflight passes, re-deriving “is another implement running” from process listings is forbidden: on 2026-08-18 three headless dispatches found their own process with ps, concluded a collision, and self-aborted with zero work done (implement/SKILL.md:126).

Why two reviewers instead of one stronger one? Because the two roles fail differently. The critical reviewer’s failure mode is missing a subtle bug, so it gets the strongest available judgment. The security reviewer’s failure mode is refusing to engage at all: the model family that is strongest at general bug-finding carries safety classifiers that can false-refuse on legitimate auth, JWT, and credential code, which is precisely the code a security reviewer must read. So the security seat is pinned to Opus in all modes, and the strongest model is deliberately kept out of it (implement/SKILL.md:269,280). That is the rationale as the code records it. Honesty requires the caveat up front: the false-refusal claim was recorded in three places with no dated incident artifact behind it, and when this audit finally tested it (section 7.1), Fable reviewed 12 of 12 auth-adjacent diffs without a single refusal. The exclusion survives on cost alone. The diagram above shows what the code says; section 7.1 shows what the measurement says.

3. The team shape: /plan-create

/plan-create turns a discovery report into reviewed, implementation-ready ticket specs. Where /implement is a serial pipeline with parallel review, /plan-create is a team: creators, reviewers, fixers, and re-checkers, phased, with workers shut down between phases so no context bleeds across roles.

• Creators (model: opus) write ticket specs in dependency waves: backend tickets before the frontend tickets that consume their APIs (plan-create/SKILL.md:218-224). A creator that discovers a missing sibling ticket may surface up to three evidence-gated discovery blocks, but only the supervising lead mints files: one writer per file, always (plan-create/SKILL.md:246-267).
• A deterministic lint gate runs between creation and review: structural problems (missing sections, invalid frontmatter) are caught by a script, not by a model, so reviewer attention is spent only on judgment (plan-create/SKILL.md:273-293).
• Reviewers (model: opus) run in parallel, one per ticket, thirteen dimensions each.
• Fixers (model: sonnet) run only on tickets scored NEEDS_FIXES, and get exactly one cycle; a ticket still failing after fix and re-review is reported for manual attention, not looped (plan-create/SKILL.md:337-345,564).
• Every wave arms a stall alarm (600 seconds, one per spawn batch), because the documented failure mode of background agents is not crashing loudly but going quiet (stalled-agent-protocol.md:12).

The halt boundary is the same one the drain honors: created tickets land at status: todo, never execution-ready. Promotion to the auto-merge queue is a human act, in both systems, always (plan-create/SKILL.md:265,434).

4. Read-only investigators, and the topology that failed

The cheapest role in the system is the read-only investigator: an agent that can search and read but not write, spawned when a question spans more files than the supervisor wants in its own context. The scoping does real work: an investigator cannot “helpfully” fix what it finds, so its findings arrive as claims with citations, and the supervisor decides what becomes a ticket.

The fan-out pattern peaks in two skills:

• /plan-explore spawns up to four parallel investigators (code search, docs, user scenarios) on sonnet, then hands the draft to a two-agent synthesis pair: an Analyst on the default model and a Critic escalated to the strongest model, because adversarial validation of a discovery draft is the highest-judgment step in the flow. The debate is capped at two rounds and the lead decides (plan-explore/SKILL.md:85,283).
• /plan-review-swarm runs five adversarial reviewer personas in parallel (security and auth, test coverage, codebase fact-checking, race conditions and state, operator validation), each capped at three findings so the synthesizer dedups signal rather than noise. The personas run on sonnet, with a recorded rationale: “Haiku is too thin for adversarial-with-grep; Opus is overkill per persona” (plan-review-swarm/SKILL.md:63,99). It has shipped as a spike since 2026-05-26 and is documented as augmenting, never replacing, the sequential review.

And then there is /debug-swarm, which this audit is obligated to report as the honest counter-example. The design is four blind investigators on opus (reproduction, telemetry, git history, hypothesis), each forbidden to see the others’ work, with a hard evidence rule: a conclusion with an empty evidence array is demoted to hypothesis, and “not reproduced” requires the exact commands and output of the attempt (debug-swarm/SKILL.md:12,121,244). The rule is good. The topology, measured, was not: on 2026-07-20 a four-investigator swarm returned zero of four result blocks, all four going idle, and the entire root-cause analysis was produced by working the angles inline. A second bug investigated inline from the start reached a sharper answer in about 15 minutes (debug-swarm/SKILL.md:56-58). The skill’s own body now steers bounded bugs inline and reserves the swarm for genuinely parallel evidence-gathering. That correction is the audit method working: the measurement beat the architecture diagram, and the diagram yielded.

5. Models chosen per risk: how assignment actually works

Model assignment in this system is not one decision; it is 33 small ones, made in skill definitions, agent frontmatter, and dispatcher code. The mechanisms worth naming:

• The dispatch pin. Every headless dispatch passes an explicit --model (default sonnet). This exists because on 2026-08-19 a bare dispatch inherited the operator’s interactive model selection and silently ran the drain on the most expensive model available, at roughly twice Opus cost, until the round errored out (bin/drain:1257-1269). The planning pipeline pins its phase dispatches identically (pipeline_runner.rb:235-239). The pin is evidence-backed. The choice of sonnet specifically is precedent (“every prior healthy dispatch had run on sonnet”), not measurement.
• The mode gate. The critical reviewer runs the strongest model interactively but drops to Opus under headless dispatch, trading peak judgment for cost on the path that runs unattended many times a day (implement/SKILL.md:264-265). The always headless fixer pipeline resolves the same gate to Opus permanently, keeping the three-attempt self-heal loop off the 2x-cost model (fixer-agent/SKILL.md:93-99).
• The security exception. The strongest model is excluded from the security seat in all modes, as described in section 2.
• The local family. The backlog-hygiene classifiers (freshness, dedup, dispatch triage) run on local open-weight models via Ollama, pinned in a code registry with the benchmark table reproduced beside the pin (hygiene_models.rb:18-73). This is the only corner of the system where model choice was measured properly: a 2026-06-01 benchmark of six models on 20 hand-labeled tickets, and a 2026-08-21 re-benchmark on a 70-entry hand-verified corpus. It is also where measurement delivered the most uncomfortable result, reported in section 7.

6. The model-assignment inventory

The audit’s core deliverable. Every model-bearing role in the system, with the evidence for its assignment or the honest entry “habit, no recorded evidence”. “Reasoned, no measurement” means a rationale is written down but nothing was ever measured against an alternative. Full citations per row are in the internal fact sheet.

Tally: 33 slots. 18 habit with no recorded evidence. 4 measurement-backed, and all four are the local models. The remainder are recorded decisions or incident-backed pins whose reasoning was written down once and never tested against an alternative.

The pattern is worth stating plainly: the cheaper the model, the better its evidence. The local classifiers were benchmarked twice on hand-labeled corpora because they were suspected of being weak. The frontier-model assignments were never benchmarked because they were assumed to be strong. Suspicion produced measurement; confidence produced habit. One caveat on the word: “habit” throughout this inventory describes the state of the record, not necessarily the state of the reasoning. Section 7 returns to the distinction.

7. The audit answer, honestly

Do role-scoped subagents beat one strong agent? The role-scoping itself is the best-evidenced part of the architecture, but the evidence is incident-shaped, not comparison-shaped. Fresh-context re-reviews, the merge-state gate, the one-writer rule, read-only investigators, the never-score-a-missing-review-as-PASS rule: each exists because a specific failure occurred and the role boundary is the repair. We would rebuild all of it. What does not exist is a controlled comparison against the alternative: no experiment ever ran the same ticket population through a single strong agent with a long context to see what the pipeline’s overhead buys. Nor was the pattern adopted blind: the operator attests it was weighed critically against what contemporary open-source orchestration frameworks were doing when it formed. That comparison lives outside this record, so this paper does not cite it as evidence; it is noted so the absence of an artifact is not misread as an absence of thought. And where a topology was measured against inline work, the topology lost: the debug swarm’s zero-of-four (section 4) is the only direct topology measurement on file, and it favored the single agent for bounded bugs. The honest position: role separation earns its keep through verifiability (each gate is checkable in isolation) rather than through any measured quality delta, and fan-out parallelism specifically is overclaimed relative to its evidence.

Are the model assignments evidence-justified? Mostly no, per the inventory. Three specific embarrassments deserve print:

1. The tiering story we tell is not what runs. The recorded pattern is “the fix agent runs one tier lighter than the implementation agent.” But the implementation agent is unpinned and inherits the dispatcher, which under the drain means sonnet: the same tier as the fix agent. The tiering only exists in interactive sessions. Nobody decided that; it fell out of an unpinned spawn call (implement/SKILL.md:194-201).
2. Reviewer value went unmeasured for three months despite being instrumented, and the first measurement (run on review of this draft, 2026-08-30) rewrote the question. The measurement found: a reviewer produced an actionable finding on 26.8% of 347 dispatches lifetime and 77.3% since August; the critical reviewer was the sole source of 28 of the 39 hard FAIL verdicts, the security reviewer of 2; and the obvious query is a trap, because the telemetry field implement_review_verdict records the verdict AFTER fix cycles and therefore reads PASS on 12 of 12 completed dispatches while the same dispatch logs carry 160 hard reviewer FAILs. The field measures survival, not catching, and its own commit message claims the opposite of what it does. The previous paper’s “fix-cycle counter read 0 on 55 of 55 rows” was itself an artifact twice over: a field-identity mistake (that counter belongs to the post-deploy self-heal fixer, and /implement’s own counter shipped only on 2026-08-28) and a telemetry partition that is 99.3% content-free noise rows, now ticketed. Two gaps survive the measurement: the Fable interactive critical path has zero telemetry coverage, and nothing can yet attribute a catch to a seat by query (the prerequisite instrumentation is ticketed).
3. The one properly measured classifier is a no-op. The 2026-08-21 hygiene re-benchmark recorded the production freshness pin at 42.9% accuracy with 0 of 40 non-fresh tickets recalled: it stamps everything fresh. The decision record calls itself “a DISPROOF, not a selection” (hygiene_models.rb:28-41). The gate machinery around it is real and fail-closed; the classifier inside it currently discriminates nothing. The docs that present the sweep as a working filter are ahead of the evidence (section 8).

Claims this paper deliberately does not make: that Fable false-refused on our security code on a specific date (asserted in three places, no incident artifact; tested on review of this draft and unsupported, see the next section), that Opus is “SOTA at bug-finding” (uncited capability claim in a decision record), and any per-seat reviewer catch rate (the field that could carry it does not exist yet).

7.1 From habit to falsifiable claims

Forcing retroactive justification onto the unevidenced rows would repeat the failure this audit exists to catch: a plausible rationale backfilled after the fact is indistinguishable from drift, and it forecloses the question instead of answering it. The operator’s direction on review of this draft (2026-08-30) is the alternative this section adopts: restate each assignment as a claim stated precisely enough to be killed, name the cheapest test that could kill it, and put the live alternatives beside it with their tradeoffs. A claim nobody has tested is not evidence. But it is honest in a way a backfilled rationale is not, because it names exactly what would change our mind. The hygiene benchmark set the precedent that a disproof is a first-class outcome; these are written to die the same way.

Run order matters more than coverage. The security-refusal corpus test is nearly free and settles a claim this audit found asserted three times with zero artifacts. The reviewer comparison attacks the two most expensive seats and rides on the telemetry work already ticketed. The rest can wait for a reason to exist: an untested falsifiable claim is still a better artifact than a backfilled rationale, because it carries its own kill condition.

8. What we believed vs what the code does

The audit rule from the previous paper applies: every mechanism claim gets checked against code before it gets printed, and the drift found becomes corrections or tickets before the paper does. Nine findings this round:

1. Drain telemetry lies about the dispatch model by default. Dispatch runs sonnet (bin/drain:1267); telemetry records an Opus-class label from a second constant with a different default (bin/drain:2282). Same env var, different fallbacks: any per-model cost slicing with the env unset is wrong. Already ticketed by the previous paper’s audit, still open.
2. “Fable is reserved for two skills only” is stale. A memory note and a comment in bin/drain:1261 both say it; the code pins Fable in six additional seats (implement’s interactive critical review, plan-explore’s Critic, copy-review, support-triage, support-reply). The “never for security review” half remains true everywhere. Memory corrected with this paper.
3. One agent definition contradicts its own call site. user-scenario-researcher pins opus in frontmatter; its primary caller spawns it with sonnet under a blanket rule (agents/user-scenario-researcher.md:4 vs plan-explore/SKILL.md:85,232). One of the two is not what runs. Ticketed.
4. The agents index lists two agents that do not exist on disk, both artifacts of retired integrations. Ticketed with the doc sweep.
5. /plan-explore still prompts for a wiki that was deprecated and tells its synthesizer to cross-reference a tracker the same skill elsewhere says to skip (plan-explore/SKILL.md:198,207,308). Same sweep ticket.
6. /doc-gen mandates “Opus 4.6”, a version not in the current lineup, and enforces the mandate nowhere (doc-gen/SKILL.md:3,14). Same sweep ticket.
7. The hygiene sweep’s docs are ahead of its evidence. Project notes present the sweep as a working freshness filter; the model registry itself records the classifier as effectively a no-op (finding 3 of section 7). The docs need the disproof stated where the confidence is. Same sweep ticket.
8. /debug-swarm’s headline oversells its own default. The description advertises four investigators; the body, corrected after the 0-of-4 measurement, steers most bugs inline. Not a contradiction, but the summary a model reads first describes the path the skill itself no longer recommends. Same sweep ticket.
9. The “fix agent one tier lighter” belief (finding 1 of section 7): true only interactively, and by accident rather than decision.

The drift pattern matches the previous paper’s diagnosis exactly: code corrects at incident speed, prose corrects at documentation speed, and memory hooks correct slowest of all. It is why this series verifies against code, not against our own notes, and why the inventory’s “habit” entries are labeled as habit instead of being backfilled with plausible rationales.

9. Open problems

• The claims table got its first result the day it was written, and the result was humbling in the right direction. The reviewer measurement ran on review of this draft. It killed one published number (the previous paper’s 55-of-55, corrected there), validated the critical-reviewer seat, exposed a 99.3%-noise telemetry partition, and proved the per-seat question needs one instrumentation change before any model decision is defensible. The security-refusal corpus test ran the same day and killed the claim outright (0 of 12). Next kills, in order: the first-pass verdict field, then the fable-vs-opus and opus-vs-sonnet reviewer comparisons the new field enables. A claims table nobody ever tests degrades back into habit with better formatting.
• A rationale can outlive its evidence by two months and be repeated in three places without anyone checking. The false-refusal claim was recorded on 2026-07-01, copied into two skill definitions and a memory note, and cited as settled in the fact-gather for this paper. Thirteen dollars disproved its current basis. The open problem is not that one claim was wrong; it is that nothing in the system flags a rationale that has never been tested, and the claims table is a manual substitute for that.
• Something moved the fix-cycle rate five-fold at the end of July, and we do not know what. 14.9% of dispatches ran a fix cycle before 2026-08-01; 77.3% after, with the 2-cycle cap now the modal outcome. Reviewer models were constant across the boundary. The step localizes to roughly 07-29 through 08-01, a window holding both a pipeline hardening batch (including the stalled-reviewer enforcement gate) and the start of a harder feature batch, and the four dispatches that could separate those explanations cannot carry an attribution. Until this is understood, no fix-cycle trend is interpretable as reviewer behavior alone.
• The freshness classifier’s future is an open decision: move the task to a frontier model, retire automated stamping, or accept the no-op as a cheap TTL refresher. The 2026-08-21 decision records the question; nothing has answered it.
• Fan-out ergonomics remain the weakest harness surface. The 0-of-4 idle failure, the stall alarms, the unnamed-synchronous workaround: all compensate for background agents that fail quiet. The system’s topology choices are partly downstream of that reliability gap, and would deserve re-audit if it closes. This paper’s own fact-gathering and refusal-corpus agents both went idle after finishing their work and before writing it up; the supervisor re-derived the results from their artifacts. The pattern the paper describes happened to the paper.

10. References

• The companion fact sheet (internal): every mechanism claim above with its full citation, compiled by reading the skill definitions, agent definitions, and dispatcher source end to end before this paper was written.
• Internal measurement notes: the reviewer catch-rate measurement (2026-08-30, 347 dispatches from forensic logs, fixture-validated) and the Fable security-refusal corpus test (2026-08-30, 12 diffs, every output verified from raw JSON), both quoted with their sample sizes and limits in section 7.
• Internal decision records: the two local-model benchmarks (2026-06-01 and 2026-08-21) behind the only measured rows in the inventory.
• The Drain: the dispatch layer under this paper, and the trust-ladder frame it borrows.
• Files as Prose, Store as Truth: the substrate both run on.
• The machine itself: the implement, plan-create, plan-explore, debug-swarm, and plan-review-swarm skills, the agent definitions, the dispatcher, the pipeline runner, and the model registry, private, at HEAD 2026-08-30.

This is a paper in the Yianna & Claude Whitepapers series: working papers on running a production side project at the output of a team, with a fleet of AI agents, alongside a full-time engineering job, written from the system’s own measurements and incident reports. Evidence citations reference our internal decision records and postmortems; where a claim could not be traced to a measurement, the text says so.