Lesson 26. Seven Smaller Published Factories and What Each Adds

On the .NET runtime repository, Microsoft's maintainers handed issues to a coding agent for ten months. At first the agent wrote code it could not compile, and only 38.1% of its pull requests were accepted. After the team rewrote its instruction file and fixed the sandbox network allow-list, that figure rose to 69% (Microsoft .NET team, first-party measurement, observed 2026-10-28). Then a second problem appeared. One person assigned the agent nine tasks at once, each pull request needed five to nine hours of expert review, and three were still open a week later. A reader who copies the headline percentage and skips that paragraph inherits the review pile-up. Datadog tells the same kind of story from the other side: its earlier AI review tools behaved like noisy linters and engineers learned to ignore them (Datadog via OpenAI's site, vendor-hosted, observed 2026-10-28). Each smaller factory publishes one useful mechanic and one caveat, usually in different paragraphs. Decide now: when you read nine company posts, what exactly do you carry back to your own repo: the number, the architecture, or something you can test on Monday?

This lesson gives you factory/adopt/small-flows.md - one borrowed mechanic per company, each with a test. It serves the Prove stage of the course loop: these write-ups feed your factory charter only after each borrowed idea has passed a check in your own repository.

What is actually going on

Nine write-ups fall into two shapes. Seven describe a line from ticket or chat thread to pull request: Microsoft's .NET team, Atlassian, Block, Meta, Linear, Datadog and Cursor. Two describe migrations, Airbnb and Google, where there is no ticket and the unit of work is a file or a code area. Most of these posts are thin on gates and each is strong on one mechanic. None of them runs every mechanic below, so the picture is a map of where each idea sits, not a company's pipeline.

Read the diagram top to bottom: the generic line has seven positions, and each company contributes at one of them. The migration pipelines enter at the end, at the human approval, with their own per-unit checks upstream.

The diagram shows why a roundup is useful: the mechanics are complementary, but you will adopt them one at a time, each at its own position.

Microsoft .NET runtime

A maintainer assigns an issue to GitHub's Copilot coding agent. It works in a sandboxed Linux runner behind a firewall allow-list, reads a repo instruction file, builds the baseline first, and opens a pull request. Copilot review runs on every PR, human or agent. Humans iterate through comments and merge (Microsoft).

Atlassian

In vulnerability fixing, Jira Automation starts a dispatcher before the working day. It marks each ticket eligible, groupable by shared dependency, or manual. A coding agent follows decision-tree instructions kept in the repo, runs the full build and tests, and stops with a report when checks fail or the fix is outside the known patterns. Human review decides the merge, then a closer pipeline checks that the deployed build holds the commit (Atlassian). Separately, Rovo Dev reviews every PR first and reads acceptance criteria from the linked Jira item (Atlassian).

Block

An engineer tags Builderbot in Slack, or it picks up a Linear or Jira ticket. Agents research, plan, implement, open the PR, then watch CI and iterate on failures. The post says humans step in where they add most value but names no approval step (Block).

Meta Forum

A machine-readable spec feeds a loop of plan, implement, build, test and correct. Verifiers sit inside it: build status, unit tests, UI matching, and navigation agents driving the live app. Hooks lint after each turn and validate tests before a PR opens. An engineer signs off every shipped outcome (Meta).

Linear

A Datadog monitor files an issue into triage and automation assigns it to Linear Agent. The agent must show it understands the problem before attempting a fix; if it cannot fix, it posts its investigation anyway. A coding agent reviews each PR and a human gives final approval (Linear).

Datadog

A pilot auto-reviewed every PR in one large repo with Codex while engineers voted thumbs up or down. A replay harness then rebuilt the PRs behind past incidents, ran the reviewer as if at original review time, and asked incident owners whether the feedback would have mattered. Humans keep architecture and design review. The account is hosted by OpenAI (OpenAI).

Cursor

A cloud agent works in its own VM, built from a Dockerfile that mirrors the developer setup, with egress limits and secret redaction. It tests end to end and attaches video, screenshots and logs. Humans review those artifacts and the PR. A "Cloud Doctor" automation opens PRs that fix broken environments and skills (Cursor).

Airbnb (migration)

Each of thousands of test files is a state machine: refactor, then Jest, then lint, then type check, advancing only when the previous check passes. A failed step retries with the latest file and the exact errors, up to a cap. Engineers finish the last few percent by hand (Airbnb). In a separate talk on everyday agent use, speakers describe the author reviewing before a PR opens and a second human reviewing the PR (Airbnb developer platform, transcript read, not watched).

Google (migration, 2024)

An expert picks the migration and a tight superset of files with Code Search and scripts. A toolkit produces changes that must pass unit tests, the engineer fixes failures, and the change is sharded to the owners of each code area for review. This is a 2024 post and stale as a description of today's flow (Google).

The rule and its condition

Use a company's mechanic when you can name the position on your line where it would sit and write a test that passes or fails in your own repo. Otherwise do not borrow it; file it as reading. Borrow the mechanic, not the number.

The condition is checkable in a minute: can you say, for the mechanic, which of your gates changes and what output proves it worked? If the only evidence you can cite is the company's percentage, you have a slogan and not a mechanic.

The first table covers the four companies with the fullest flows. Cells marked "not published" are absences in the sources read, not claims that the company lacks the thing.

Sources for table one: Microsoft, Atlassian (single-team numbers in a post promoting a beta product), Block (newsroom post; the 15% is of production code changes, method unpublished), Meta (small greenfield team), all observed 2026-10-28.

The second table covers the remaining five, including the two migrations.

Sources for table two, all observed 2026-10-28: Linear (vendor, anecdotal), Datadog on OpenAI's site (vendor-hosted, secondhand), Cursor (vendor, PR counts not defects), Airbnb (self-reported), Google (self-reported, stale).

What these flows do well: each pairs an agent step with a check it does not control, and several end in a human who reads. What they cost: the figures count different things (PRs, vulnerabilities, files, incidents), so no column adds up to another, and none of the nine records a cost per merged PR.

Worked case

The row of Cursor's table that says "over half" is the kind of line that ends up on a board slide. Decide now: your manager asks which of the nine posts proves that agents will work at your scale. Do you answer with the strongest number, or with a short list of mechanics you will test?

None of the nine proves it. The numbers tell you where to look, and your own run tells you whether it holds. The rest of this section extracts the one mechanic from each company, with its figure's provenance, its caveat, and the lesson of this course it feeds.

Microsoft: prepare the sandbox before judging the agent. The success rate rose from 38.1% to 69% after instruction and firewall fixes, and the revert comparison was 3 of 535 merged agent PRs against 33 of 4,251 other PRs, which the post itself calls not statistically meaningful. Those counts are first-party and the tasks were chosen by authors, not randomised; the revert search was by title and may undercount (Microsoft, observed 2026-10-28). Feeds Lesson 9, readiness before autonomy.

Atlassian: every ticket gets a recorded outcome, and a closer checks the deploy. One team reports 120+ vulnerabilities resolved and 95% of its agent PRs merged on the first run, with the denominator undefined (Atlassian, self-reported, one team, observed 2026-10-28). Its separate first-reviewer post claims a 45% cut in median PR time with the method unread (Atlassian, vendor product, observed 2026-10-28). Feeds Lesson 13, context the repo cannot give, and Lesson 10, where the shipper owns the rollback.

Block: the run does not end when the PR opens. About 1,500 merged PRs a week and about 15% of production changes are Block's own claims, with no method and no named gate (Block, self-reported, observed 2026-10-28). Take only the CI-watching loop, and add the round cap from Lesson 4 yourself; Block did not publish one.

Meta: verifiers live inside the loop. Hooks lint after each turn and validate tests before a PR exists; the headline is 8 weeks to parity with Facebook Groups for a small greenfield team (Meta, self-reported, observed 2026-10-28). It publishes no merge or defect rate. Feeds Lesson 8, deterministic nodes around agent steps.

Linear: show understanding first, and leave a diagnosis if you cannot fix. No volume is published; dead flags are fixed nearly every time and messy failures about a third of the time, by an engineer's anecdotal account (Linear, vendor, observed 2026-10-28). Feeds Lesson 13.

Datadog: test the reviewer on your own incident history. Ten or more cases, about 22% of incidents examined, where feedback would have changed the outcome, highest among the tools compared (Datadog via OpenAI, vendor-hosted, observed 2026-10-28). Feeds Lesson 4, the review gate.

Cursor: review recorded proof, not only diffs. Cloud agents authored about one in ten merged monorepo PRs in December 2025 and over half by 30 July 2026, a seven-day rolling chart that counts PRs, not defects (Cursor, self-reported vendor, observed 2026-10-28). Feeds Lesson 9, because the environment is the product.

Airbnb: one validator per state, retry with the latest error. 75% of files in four hours and 97% after four days of tuning; the six-weeks-versus-one-and-a-half-years comparison is a projection (Airbnb, self-reported, observed 2026-10-28). No dollar cost is published. Feeds Lesson 8.

Google: only test-passing changes reach owners. 80% of landed modifications in these migrations were AI-authored and engineers estimated about half the time saved, in a 2024 post (Google, self-reported, stale, observed 2026-10-28). Feeds Lesson 13, ownership as routing.

Now the deliverable. Fill it for one real repo. The mechanic column is prefilled; you supply where it lands, the test, the pass condition and an owner.

awk -F'|' '/^\|[-: |]+\|$/ { next }/^\| *(Field|Company) *\|/ { next }/^\|/ {  rows++  for (i = 2; i < NF; i++) {    c = $i; gsub(/^ +| +$/, "", c)    if (c == "" || c ~ /^</ || c ~ /^TODO/) { printf "line %d, cell %d: empty or placeholder\n", NR, i - 1; bad = 1 }  }}END { if (rows < 12) { printf "expected 12 table rows, found %d\n", rows; bad = 1 } exit bad }' factory/adopt/small-flows.md

Faded case

Skip to the open case if this is familiar. One decision is left to you.

First, how the group sits against three companies from earlier pages of this module.

Sources, all self-reported by the company and observed 2026-10-28: Stripe, Cloudflare, Intercom.

Illustrative composite. A four-person team runs an agent on a Java service. Over the last month most agent PRs were closed unmerged, and the review comments on them say "does not build" or "tests were never run". The reviewer queue is short. The team may adopt one mechanic from this lesson this week. Decide now: which one, and why not the others?

Open case

This section is the course's judgment, not any company's advice.

Solo developer. Copy Microsoft's preparation (instruction file written from the last failed runs, a baseline build before any change) and Linear's confidence gate on one narrow bug class. Skip Cursor's cloud infrastructure and Block's orchestration; you are the only reviewer, so a bigger line only moves work to you. First week: write the instruction file from your last three failed agent runs, then run the Microsoft test from the template.

Small team. Copy Meta's hooks and Atlassian's per-ticket outcome. Skip incident replay until you have a handful of incident PRs to replay. First week: add a pre-PR hook that runs the tests, and log one outcome line per ticket the agent touches, including "stopped, here is why".

Enterprise. Copy Datadog's replay for reviewer evaluation and Google's owner sharding for migrations. Skip cross-company scale figures as targets. First week: list last quarter's incident-causing PRs and replay them through your reviewer, with incident owners judging.

Regulated shop. Copy Atlassian's recorded outcome and closer, and Airbnb's per-unit validators for any migration. Skip no-human approval and the share-of-AI-code figures. First week: decide where outcome records live so an auditor can query them, and run the Atlassian test on ten tickets.

Illustrative composite. A payments team of twenty-five runs an AI reviewer on every PR. Engineers ignore its comments, and nobody can say whether it would have caught anything that mattered. The auditor will not let its comments count toward a required check without evidence. You can adopt exactly one mechanic from this lesson this quarter. Which one, and what do you show the auditor?

Where this breaks

The numbers are weak evidence. Almost every figure is self-reported, vendor-hosted or both: Datadog's result appears on OpenAI's site, Cursor and Atlassian describe products they sell, Block's is a newsroom post, and Google's is a 2024 post that says nothing reliable about the current flow. Treat each figure as a pointer to where a mechanic sits, never as a target.

Microsoft's counts are the closest to measured, and they still cover tasks that authors chose, not a random sample. The revert comparison is described by the post as not meaningful statistically, so "the agent is as safe as humans" is not a conclusion it supports (Microsoft, observed 2026-10-28).

The mechanics are not additive. A confidence gate, a replayed reviewer and a recorded-artifact check all add latency and review work; the Microsoft case already shows review capacity becoming the ceiling. Add one, measure, then add the next.

What would change the verdict: Block naming its approval step, Meta publishing merge and defect rates, Datadog publishing its own account with the incident count, and any company publishing cost per merged PR with a denominator. Until then, the safe use of this page is tests in your own repo.

Key Takeaways

• Borrow the mechanic, not the number.
• Write a test for every mechanic you copy, and run it in your own repo.
• Prepare the sandbox and instruction file before you judge an agent's merge rate.
• Replay your own past incident PRs through a reviewer before you trust its comments.
• Label every borrowed figure with its provenance: self-reported, vendor-hosted or stale.
• Check units before comparing: PRs, files, vulnerabilities and incidents have different denominators.

What You Have Now

factory/adopt/small-flows.md - one borrowed mechanic per company, each with a test. It sits in the factory/adopt/ folder beside the larger company pages in this module. Each row points back to a gate you already own: the sandbox preparation to Lesson 9, the review replay to Lesson 4, the confidence gate and ownership routing to Lesson 13. The validator command gives you a one-line way to stop the file going stale with blank cells. It serves the Prove stage of the course loop: the file is evidence for the factory charter, and a row without a passing test is simply not in it.

Further Reading

• Microsoft .NET team, "Ten months with the Copilot coding agent in dotnet/runtime": the fullest first-party counts, with their own caveats on method.
• Atlassian, "Vulnerability fixes with agentic pipelines": dispatcher, agent and closer with a recorded outcome per ticket.
• Meta Engineering, "How Meta built Forum, AI-native engineering": hooks and verifiers inside the loop, and the failure of an agent that lost its place.
• OpenAI, "Datadog": the incident-replay method, vendor-hosted.
• Cursor, "Cloud agent environment": environment as the product and recorded artifacts as proof.
• Airbnb, "Accelerating large-scale test migration with LLMs": a per-file state machine with a validator per state.

Exercises

1. Fill the template. Complete the "Where it lands" and "Passes when" cells for all nine rows in your repo. Mark every row where the position on your line does not exist yet.
2. Run two tests. Run the Microsoft baseline-build test and one other sample test from the template. Record pass or fail and what you changed because of it.
3. Pre-register a threshold. Choose one row, write the pass threshold before running its test for a week, then write down what the result tells you and what it does not.

Check your understanding