Lesson 22. Uber: Cost Per Session Down, Cost Per Merged PR Still Unpublished

In August 2026 Uber's engineering blog told an unusually open cost story. Agent use had grown fast: weekly active users rose 7x and weekly agentic requests 9.4x between February and August, while total AI spend stayed roughly flat since April (Uber, self-reported, observed 2026-10-28). With the model held fixed, cost per session fell 52% from its June peak (same post). That matters because AI-related cost had climbed about 6x since 2024 (Pragmatic Engineer, reporting Uber, observed 2026-10-28). Yet the post also says each managed agent is tracked on cost per merged PR and on a revert rate, and it prints neither value. A session can get cheaper because the harness improved, or because sessions got shorter and did less. The falling line cannot tell you which. A finance lead who reads it learns that the meter slowed, not that a shipped change got cheaper. Decide now: if you ran this factory, which cost number would you put on the wall, and what would you need to collect before you could print it?

This lesson gives you factory/adopt/uber.md - an evidence-table for reviewers and a cost metric plan. It serves the Prove stage of the course loop (Slice, Gate, Run, Wrap, Prove): the stage where a published case study becomes numbers you hold your own line to.

What is actually going on

Uber's public material describes a line that spends effort before it spends shared CI. A request can arrive from Slack, the web, a GitHub PR or a CLI, and a cloud agent called Minion picks it up (Uber speaker, recorded talk, observed 2026-10-28). In the AI Engineer 2026 demo, an assistant first researches the idea and writes a spec with design variants. Minion then works inside a pre-provisioned DevPod, with the monorepo and internal tools in reach (Uber talk page, observed 2026-10-28).

The unusual step comes next. The draft PR is deliberately held back from CI. Inside the agent's own loop it fixes static-analysis findings, launches the app in a simulator, compares a screenshot against the Figma design, and starts the backend in staging for an integration check. Only a PR that survives this opens to shared CI. There, self-healing CI repairs failures, a small model reviews early and a larger reasoning model reviews later (same talk page). The PR arrives carrying a table of the checks performed, screenshots included, so the human reviewer starts from evidence instead of from nothing.

Code Inbox then decides who looks at it. It gathers the AI review results, sanity-test screenshots, risk scores and blast radius for each PR, which replaces first-in-first-out review (Uber speaker, recorded talk, observed 2026-10-28). The blog adds that an automated reviewer, uReview, runs on every PR (Uber, observed 2026-10-28).

Three labels keep this honest. First, the demo is a proposed workflow, not a shipped feature, and the speakers say the components sit at different maturity. Second, who approves the merge is not in the blog; a podcast summary says a human still approves every merge, and that auto-landing "green lanes" wait on back-tested risk scoring (secondhand, observed 2026-10-28). Third, nothing public describes what happens after merge for agent PRs.

The diagram has one loop, between the agent and its own checks, and one terminal box that is deliberately empty. Everything the reader can verify sits before the merge.

The rule and its condition

Use Uber's shape when reviewers, not agents, are your scarce resource and a green test run is not enough proof for the change. Make the agent validate in its own loop, and attach the evidence to the PR. Otherwise, when changes are small and one required check already proves them, a plain checklist costs less and the table is overhead.

The table below uses the seven dimensions every company page in this module shares, so you can compare pages side by side.

The figures come from the Uber blog (the 70% attribution, self-reported), the recorded Minion talk (over 11%, speaker's own), the Pragmatic Engineer write-up (84%, via Uber), and a podcast summary (about 15%, secondhand). The SubmitQueue design is in an ACM paper. The Pragmatic Summit session (158,887 views, YouTube page) is excluded from every cell above, because only its chapter list was readable and the body was not. The AI Engineer talk, by contrast, had 30,609 views at observation (talk page), and we read its page text, not the raw video.

Worked case

Suppose you copy only what Uber published: spend flat, cost per session down. Your finance lead asks whether the factory got cheaper per shipped change. Decide now: can you answer from those two numbers, and what is the smallest addition that would let you?

You cannot. Cost per session is a rate per unit of effort, and a shipped change is a unit of outcome. A worked example shows how the two come apart.

Illustrative composite. A team spends $4,200 in March across 600 agent sessions, so cost per session is $7.00, and it merges 42 agent PRs, so cost per merged PR is $100 with 3 reverted, a revert rate of 7.1%. In April it spends $2,940 across the same 600 sessions, so cost per session fell 30% to $4.90, but only 28 agent PRs merge, so cost per merged PR rose to $105, with 1 reverted, a revert rate of 3.6%. The session meter says "cheaper". The merge meter says "slightly dearer, with fewer reverts". Both are true, and only the second answers the finance lead.

The answer is to divide the same spend by merged work and pair it with a quality number. Price the merge, not the session.

Here are the mechanics worth taking from Uber, each tied to a lesson in this course.

1. Validate in the agent's loop before shared CI. The cheap rejections happen where they cost least. This is a Gate move, and it sits upstream of the review gate from lesson 4.
2. Put the evidence in the PR. A table of checks lets the reviewer see what ran. It is evidence for the human and never the approval itself, which keeps lesson 4's rule that the author never holds the merge key.
3. Route review by risk and blast radius. Code Inbox is the same idea as the risk lanes in lesson 7: a small, clean change should not wait behind a risky one. A secondhand summary says auto-landing waits for back-tested scores, which is the right order.
4. Cap the maintenance loop at what reviewers can read. Scheduled cleanup skills run on a Sunday with a limit on the number of diffs, so Monday's reviewers are not flooded (Uber talk page, observed 2026-10-28). Review throughput is the ceiling, as Agent Fleets lesson 11 argues.
5. Name the unit cost and its guardrail together. Uber's own list pairs cost per merged PR with a revert rate. Lesson 14 turns that pairing into a scoreboard row with a denominator and a tripwire.
6. Compare cost only with the model held fixed. Model upgrades change behaviour, so a drop in cost per session across a model change proves nothing. Uber's 52% figure carries this condition (Uber, self-reported, observed 2026-10-28).
7. Control spend with nudges, not hard caps. Slack messages arrive at 50%, 80% and 100% of expected spend, and moving up a tier needs a manager's sign-off (Uber, self-reported, observed 2026-10-28). Loop Engineering lesson 4 covers the spend tiers this resembles.

The file below puts the first, second and fifth mechanics on paper. Part 1 is the evidence table you paste into every agent PR. Part 2 is the cost plan, with a "Value today" column that stays honest when you cannot compute a number. Part 3 is the list of claims you refuse to make.

The validator prints the line and column of every empty or unfilled cell in any table, and exits 1 if it finds one:

awk -F'|' '/^\|/ && !/^\|[-| ]+\|$/ { for (i=2;i<NF;i++) { c=$i; gsub(/^ +| +$/,"",c); if (c=="" || c=="<fill>") { printf "line %d: unfilled cell in column %d\n", NR, i-1; bad=1 } } } END { exit bad }' factory/adopt/uber.md

Tested on a scratch copy: the raw template fails and names each unfilled line, a fully filled copy exits 0, and blanking one cell in a filled copy fails again at that line. The check only proves no cell is empty. Whether a cell is true is a reviewer's job.

Faded case

Skip to the open case if this is familiar. Three companies publish a gate the agent must clear before a human reads anything. They put the gate in different places, and the table shows where.

The sources are the Uber blog, Stripe's second Minions post (self-reported, no denominator), the Coinbase post (self-reported) and the Anthropic post (self-reported). The Stripe, Coinbase and Anthropic pages are lessons 18, 24 and 16 in this module.

Only Anthropic publishes a cost per review, and a review is not a merged PR. Uber publishes the equation and the trend. Nobody in this table publishes cost per merged PR.

Now the decision. Your reviewers say they lose most of their time working out whether a change was tested. The PRs are a normal size and CI loops are fine. You can build one gate this week: Stripe's CI cap, Coinbase's plan sign-off or Uber's evidence table. Pick one and name the number that proves it worked.

Open case

These four paragraphs are this course's judgment, not Uber's advice.

Solo developer. Copy the evidence table as a PR template your agent fills, because the reviewer is you tomorrow. Skip Code Inbox and any dashboard. In week one, run the validator on your last three agent PRs and note the row you could not fill. Track one number: provider bill divided by merged agent PRs.

Small team. Copy the table and the four-row cost plan, and route reviews to whoever owns the path. Skip custom model benchmarks and spend tiers. In week one, count merged agent PRs and reverts from the last 30 days of git history, and write "not available" where you cannot.

Enterprise. Copy per-agent unit cost with a paired quality metric, fixed-model comparison and nudges in place of hard caps. Skip auto-landing until risk scores are back-tested, as Uber itself does. In week one, pick one managed agent and publish its definitions of "merged" and "reverted".

Regulated shop. Copy the filled table as an audit record kept with each PR, and the refusal list in Part 3. Skip counting the table as approval. In week one, map each table row to a control your auditor already names, and confirm a named human approves each lane.

The scenario: your finance lead shows a quarterly chart where cost per session is down sharply with the model unchanged, spend is flat, and agent-involved PRs are up. Reviewers say they feel busier. You have git history and the provider bill. Write what you compute first, from which two sources, and what result makes you pause the lane.

Where this breaks

The headline gap is the one in the title: Uber names cost per merged PR and a revert rate as tracked per-agent metrics and prints no values for either. Until those appear, the 52% drop (Uber, self-reported, observed 2026-10-28) is an efficiency gain per unit of effort and an unproven one per unit of shipped work. Everything else in the cost story is self-reported by Uber, with no outside audit.

Second, the walkthrough is partly a proposal. The AI Engineer demo shows a workflow, not a measured experiment, and the speakers say the components are at different maturity. No quality or defect metric is given for the PRs counted in the 70% figure (Uber, self-reported, observed 2026-10-28), and "attributed to agents" is undefined. The Minion share also moves with the date and the source: over 11% from the speaker, roughly 15% from a secondhand summary. We read the text of the blog but not its figures, and we could not read the body of the Pragmatic Summit talk.

Third, the green lanes, the idea that safe agent PRs land without a human, are future work, waiting on back-tested risk scores, per a secondhand summary. Do not describe Uber as running unattended merges. The 2019 SubmitQueue paper shows how Uber lands changes by testing the combined state, but it predates agents and no source ties it to them.

What would change the verdict: published values for cost per merged PR and revert rate, with denominators and a fixed-model window.

Key Takeaways

• Validate inside the agent's loop so shared CI sees only changes worth its time.
• Attach a table of checks to each PR, and never count it as the approval.
• Compare cost only across windows where the model stayed fixed.
• Pair every unit cost with a revert rate and a denominator you can name.
• Label each figure self-reported, secondhand or unread before you repeat it.
• Price the merge, not the session.

What You Have Now

factory/adopt/uber.md - an evidence-table for reviewers and a cost metric plan. You now hold a PR evidence table that names what each check does not prove, a four-row cost plan with numerators, denominators and paired quality metrics, and a refusal list for claims you cannot back. The validator keeps empty cells out of all three. Keep the file next to your scoreboard from lesson 14 and your review gate from lesson 4, and fill "not available" before you fill a guess.

Further Reading

• Uber, "Running a software factory efficiently": the cost equation, the managed-agent metrics and the spend tiers, in Uber's own words.
• AI Engineer 2026 talk page, Uber building blocks: the Slack-to-draft-PR demo, the held-back PR and the table of checks.
• The Pragmatic Engineer, "How Uber uses AI for development": adoption figures and the cost trend, with the reporter's framing.
• Uber SubmitQueue, EuroSys 2019: the pre-agent paper on landing changes by testing the combined state.
• GitHub docs, managing a merge queue: the platform feature that applies the same idea to your repo.
• Stripe, Minions part 2: a contrasting first-party flow with a capped CI loop.

Exercises

1. Fill Part 1 for the last agent PR you merged. Mark every row you cannot fill with "not run", and say which check you would add first.
2. Compute cost per merged agent PR and revert rate for your last 30 days. Write the numerator, the denominator and the model in use. Write "not available" in any cell you cannot support.
3. Write a tripwire for pausing a lane. Combine cost per merged PR with the revert rate and review minutes, then test it against the last two months of your data and report whether it would have fired.

Check your understanding