Lesson 03. Set Autonomy Per Lane, and Know When Not to Go Dark

A small team at HumanLayer ran its codebase lights-off for about four months in 2025. Specs and tickets went in, background agents took the small and medium jobs, and nobody read the output closely. Then the outages came. One bug the agents could not fix sent the humans digging through months of code none of them had read, and after a third incident a cofounder spent two weeks rewriting patterns by hand (Horthy, HumanLayer, observed 2026-10-28). Read that with its limits in view: one team, a postmortem written by a participant, and an author who sells tools for human and agent collaboration and says so. The shape still matters. The team applied a single setting to every kind of change, whether it was a typo in a help page or a rewrite of the code that decides what users see. A setting that fits one of those cannot fit the other. Decide now: for the changes your agents merged last month, which ones would you let land with nobody reading the diff, and what written thing would justify that?

This lesson gives you factory/autonomy-map.md - each lane of your repo with its autonomy level, the check that earns it and the revert path. It serves the Slice stage: before work is cut into tickets for agents, you decide how much of each part of the repo may run without a human reading it.

What is actually going on

The postmortem made a granularity error. It set autonomy once, for the whole codebase, when autonomy belongs to a lane: a slice of the repo defined by a path (docs/, billing/) or by a change type (an additive migration, a dependency patch). Two lanes in the same repo differ on three properties. Verification strength: is there a written check, stored in the repo or CI, that can fail on the risk this lane carries? Blast radius: what is the worst thing one bad merge can reach? Reversibility: how many steps, and which side effects, does an undo take? Those three set the level. The team's mood and the model's benchmark score do not.

Other writers reached the same cut. Addy Osmani's essay says to choose dark or light per loop, not per organization, and warns that unread output piles up as comprehension debt (Osmani, secondhand essay, observed 2026-10-28). The Thoughtworks retreat reports recommend risk-tiered autonomy per system, judged on risk, reversibility and blast radius (Thoughtworks, practitioner consensus without outcome data, observed 2026-10-28). The diagram shows one repo producing three different levels from the same three questions.

No lane was ranked by importance. Each exits where its own check and its own revert put it.

What does an earned dark lane look like? StrongDM is the only first-party dark-factory showcase in the evidence this course read, and it publishes mechanics only. Acceptance scenarios live outside the codebase so the coding agents cannot see them, validation is a satisfaction score over many runs rather than a green suite, and cloned versions of third-party services let scenarios run at volume (StrongDM, self-reported, observed 2026-10-28). Its founding rules bar human code review and suggest spending at least $1,000 in tokens per engineer per day (StrongDM, self-reported, relayed by Simon Willison, observed 2026-10-28). It publishes no defect rates, incident data or cost totals. The dark there rests on a validation system built first.

Anything weaker is a guess. A preprint on seeded bugs found the best review agent caught the bug in its top five comments 71.9% of the time, which leaves roughly three in ten missed, and the share of bugs visible from the diff alone fell from 46.1% on public repositories to 22.8% on a company's proprietary ones (Yang et al., preprint, not peer reviewed, observed 2026-10-28). "Another agent reviewed it" does not earn a lane the dark.

One more data point, thinly sourced: an independent builder who ran a public dark-factory experiment posted a video dated 2026-10-16 saying that the epic-in, production-out version does not work, and describing a replacement with a manager agent, files the factory can never edit and a staging gate. The source is the video's description and chapter titles, with no transcript, so treat the details as unverified (video, observed 2026-10-28).

The rule and its condition

Use a level per lane, set from verification strength, blast radius and reversibility, instead of one level for the repo. Go dark on a lane when a written check fails on the risk the lane carries and the revert is one cheap, tested step. Otherwise keep a human at the merge: light when a check exists but the revert or the blast radius is costly, human-led when no written check exists at all.

The names are vocabulary, not a ladder you must climb. Dan Shapiro's ladder runs from a human writing every character, through intern, junior buddy and manager, to a project-manager stage and finally a dark factory, and it describes how much code a human reads (Shapiro, anecdotal taxonomy, observed 2026-10-28). Sources number the steps differently, so use the names. Your map needs only three. Dark: merges when its check passes, and no human reads the diff. Light: the agent opens the change and a named human approves. Human-led: a human writes and the agent assists.

Being strict by default is unglamorous but defensible. In telemetry from a vendor of engineering metrics, the highest-AI-adoption periods showed incidents per PR up 242.7% against the same organizations' lowest-adoption periods (Faros, vendor report, vendor-selected and correlational, observed 2026-10-28). That is a direction, not a verdict.

Do not go dark when any of these holds:

• No written check fails on the lane's main risk, or the only check is another agent's opinion.
• The revert needs a person: a refund, an apology email, a data restore.
• Nobody will sample-read the merged output later.
• You cannot name the lane's owner.

Two guards complete the rule. A change that touches two lanes takes the stricter level, and a path that matches no lane is human-led. The flowchart turns the rule into questions you can answer from your own repo.

Every exit except the last keeps a human on the lane. Use the lowest level that clears the lane's work, which is the least-agentic-pattern instinct of AI Solutions Architect, lesson 7, applied to a path instead of a product. Keep one line from all of this: A lane earns the dark with a check that can say no and a revert that can say undo.

Worked case

Illustrative composite. Tallybook is a made-up six-person invoicing product, and every number in this case is modeled, not measured. Its repo has three top-level paths. docs/ is the public help site: CI builds it, runs a link checker and a spell-check, and a redeploy takes about two minutes. app/ is the customer web app: roughly 400 unit tests plus contract tests against the billing API, covering about 60% of lines, with a rollback to the previous build in about five minutes. billing/ computes invoices and calls the payment provider: it has unit tests, but nothing replays past invoices, and a wrong run charges real customers and sends emails.

The team lead wants agents to take over all three paths and has three proposals. A: dark everywhere, because the agents are good and review is slow. B: light everywhere, with a human approving every change. C: a level per lane, set from the three questions. You have one afternoon, and the map you write today decides what the agents may do tomorrow.

Decide now: which level does each lane get, and what breaks first under the proposal you reject?

Proposal A breaks first, and it breaks in billing/. That lane has no written check that fails on a wrong total, and its revert is not a redeploy: it is refunds, correction emails and a ledger fix, all done by a person. That is the HumanLayer pattern with a payment provider attached. Proposal B breaks slower. Typo fixes and billing diffs share one review queue, so a human spends attention evenly on things that deserve it unevenly, and the review gate becomes the stalled queue Lesson 01 told you to look for first. Proposal C is the build.

Walk each lane through the flowchart. docs/ has a written check (build, links, spell-check) plus a path guard that fails the change if it touches any file outside docs/. The revert is one commit and a redeploy that runs itself, and the worst bad merge is a wrong sentence. All three answers pass, so it is dark. app/ has checks, but coverage is partial and users see a bad build before the rollback lands. That is a check without a revert you would call cheap, so it is light. billing/ fails the first question: no written check can fail on a wrong total. It is human-led, with a stated condition for moving up, because Loop Engineering, lecture 2 asks whether a task is worth looping at all, and a rare, high-stakes lane may never pass that test. Agent Fleets, lesson 16 covers how to roll back; here you only record which revert applies to each lane and whether anyone has ever tried it.

A one-line validator prints any dark lane whose check or revert cell is empty or says none, so a clean run prints nothing.

awk -F'|' 'tolower($4) ~ /^ *dark *$/ && (tolower($5) ~ /^ *(none.*)? *$/ || tolower($6) ~ /^ *(none.*)? *$/) {print "DARK LANE WITHOUT CHECK OR REVERT:" $2}' factory/autonomy-map.md

Keep the file where the agents cannot write to it. A level an agent can raise is not a level. The cost of the rejected proposals is the outage in A and the stalled queue in B. The cost of C is an afternoon, paid once, and an edit whenever a new check lands.

Faded case

Skip to the open case if this is familiar. Tallybook adds a fourth lane, migrations/, and splits it by change type. Additive migrations create a table or a nullable column. Destructive migrations drop, rename or retype something. CI applies every migration to a full-size copy of last night's snapshot and fails if it errors or the app's test suite breaks. Down scripts exist for additive migrations, but CI never runs them. The revert for a destructive migration is a restore from the nightly backup, which loses every write since the snapshot. The lane has one owner, the engineer on call for the database, and she has never run a down script outside her laptop. Nobody has timed a restore either, so no one can say whether it takes ten minutes or three hours. Tallybook's invoices and customer records live in these tables, and the lead is tempted to treat the whole lane as one row because splitting it feels like paperwork.

Using the flowchart, write the two rows' levels, and name the single change that would move the additive row up one level.

Open case

Illustrative composite. The product below is made up.

A clinic-booking product keeps one repo with five top-level paths. docs/ holds the staff help pages; CI builds them and checks every link. reports/ holds read-only queries that produce a weekly PDF for clinic managers; CI runs them against a fixture whose totals were computed by hand, and an older version can regenerate any PDF. web/ holds the patient-facing booking screens; it has unit tests and screenshot diffs for the main flows only. auth/ handles patient login and sessions; it has tests, and a bad release can lock patients out or expose records. infra/ holds the Terraform that applies straight to production; the plan is printed in CI and nobody has written any assertion on it. The agents are ready to take over everything.

Write the five rows of factory/autonomy-map.md: lane, match, level, the check that earns it, the revert path. Mark which lanes may go dark today, and for every other lane name the one thing that is missing.

Where this breaks

The evidence is thin in both directions. The only first-party dark-factory showcase, StrongDM's, publishes mechanics and no defect or cost numbers. The case against rests on one team's postmortem by a tool seller, a vendor correlation and a preprint about reviewers. Thoughtworks retreat attendees reported dark factories as viable for transactional backends and COBOL rewrites, though that is anecdotal and conducted under Chatham House rules (Ordog, retreat reflection, observed 2026-10-28). Nobody has shown whether models released after July 2026 change the postmortem's outcome. So the rule is a conservative default, not proof that dark cannot work.

Levels and headline metrics mislead. Shapiro's numbering varies by source and the ladder is a taxonomy, not evidence that higher is better. Google reported that 75% of its new code was AI-generated and approved by engineers, with the definition undisclosed (Google, self-reported, observed 2026-10-28), and a secondhand report says its CEO later called the share meaningless. Treat it only as a reminder to track outcomes per lane, such as changes landed and rollbacks, not the share of AI-written code.

A map decays, and small repos may not need one. Checks weaken as code changes, and a lane matched by path mis-covers a change that spans paths. If every path in your repo shares one risk and one revert, one lane is honest and the map is a single row.

Key Takeaways

• Set autonomy per lane, matched by path or change type, never per company.
• Choose a lane's level from check strength, blast radius and revert cost, not enthusiasm.
• A lane earns the dark with a check that can say no and a revert that can say undo.
• Reject an agent's opinion as the only check; write the check into CI.
• Take the stricter level when a change crosses lanes, and default unmatched paths to human-led.
• Treat level names as vocabulary; the evidence for dark factories is mechanics, not outcomes.

What You Have Now

factory/autonomy-map.md - each lane of your repo with its autonomy level, the check that earns it and the revert path. It lives in your factory/ directory beside line-map.md from Lesson 01 and release-slice.md from Lesson 02: the line map names the approvers, the release slice tags each issue with a lane, and this file says how much of each lane may run unattended. It serves the Slice stage. Re-read it whenever a lane gains a new check or a new revert, and edit it as a dated change made by a human. Later lessons add the gates that enforce the levels you wrote here.

Further Reading

• Osmani, "Software Factories, Light and Dark" - sets the per-loop choice between dark and light and names comprehension debt.
• Shapiro, "The Five Levels" - the levels vocabulary, with its own anecdotal footing stated.
• Thoughtworks, Future of Software Development retreat key takeaways - risk-tiered autonomy by risk, reversibility and blast radius.
• Yang et al., CRBugBench preprint - the first measured test of agents reviewing agents, with limits.
• StrongDM, Software Factory - the mechanics behind a first-party dark factory, without outcome data.
• Horthy, "Why Software Factories Fail" - the lights-off postmortem, read with its declared bias.

Exercises

1. List your repo's top-level paths and its recurring change types. Write at least four lanes with a Match column, and give each a level using the flowchart.
2. For every dark or light lane, write the check as a command CI runs, then run it against a known-bad change and confirm it fails. Time one real revert and write that revert path as commands.
3. Take last month's merged changes. Assign each to a lane, count how many would have gone dark, and for each of those find the check that would have failed on a bad version. Split any lane where the answer was "none".

Check your understanding