Lauren Tan: How Grok Bot Ships 11,000 AI PRs a Month Without Slop

Lauren Tan explains how the Grok Bot team at SpaceXAI ships thousands of AI-written pull requests while code quality improves, using lint rules, architecture and verification instead of prose. In “All Code Is Slop (Except Mine)”, a 17-minute talk on the AI Engineer channel, Tan (known online as poteto) says she personally shipped 3,000 pull requests to production last month, and her team shipped 11,000 with a revert rate of about half a percent. Her argument is that none of this comes from smarter agents. It comes from an environment built so the agent does the right thing by default.

TLDW

Tan describes an inverted “trust pyramid” for agent environments: deterministic constraints in the codebase first, then lint rules, types and tests, then runtime and formal verification, and only a thin layer of prose rules on top. Her team built Dune, a framework for Electron apps that bans error-prone patterns like React’s useEffect, forbids imports across process boundaries, and bans nearly all code comments. Verification (screenshots, videos, TLA+ model checking, Lean proofs and property-based UI tests) now replaces most code review. The payoff is a release every day, 227 contributors in October, a falling revert rate and an app that got faster as the pace rose.

Thoughts

Banning code comments sounds like a stunt until you hear why. Tan found that agents used comments, docs and rules files as a place to write down their justifications for not doing the right thing. At Cursor, agents patched over a clash between VS Code’s imperative style and React’s declarative style with setTimeout and requestAnimationFrame, then left comments explaining why each workaround was fine. The next agent read those comments as the established pattern, timers spread through the app, and garbage collection pauses made it janky. A comment is persuasion without enforcement, and agents are very good at being persuaded by text. The replacement is the clever part: a lint rule whose error message tells the agent to fix the root cause. The “why” still exists, it just lives in a place that fails the build instead of a place that can be ignored. That is a useful design principle for any team using coding agents, comment ban or not.

The deeper point is about economics. Tan notes that nothing in this stack is new. TLA+, property-based testing, proof assistants and full UI fuzzing have existed for years; most teams skipped them because they were hard to write and expensive to maintain. Agents collapse that cost, which is why she calls this “a golden age of software testing and verification” and why she says most of her tokens now go to checking work rather than producing it. That inverts how most people budget for AI coding. If generation is cheap and trust is the bottleneck, the smart spend is on the verification layer, and the human job shifts from reading diffs to reviewing videos, screenshots and state charts that show whether the agent understood the intent.

The headline numbers deserve a careful read. The 11,000 pull requests are a team figure across about 227 contributors in October, many of them casual drive-by contributors, and the title’s personal number is 3,000. A 0.5% revert rate is encouraging, but on a team that also auto-merges agent-written fixes, problems can be fixed forward instead of reverted, so the rate alone does not prove quality. The more convincing evidence is the one Tan says she finds most interesting: startup, typing, opening chats and switching chats all got faster while the volume rose, in an Electron app where slop usually shows up as lag first. Performance that improves under that pace is hard to fake.

Her closing warning is the most practical advice in the talk: if your inner coding loop produces slop, automating it just produces more slop, faster and at a higher token bill. Plenty of teams are rushing to build “software factories” of parallel agents on codebases full of workarounds and deprecated patterns, which her gardening metaphor explains well. Agents copy whatever looks like the house style, so weeds spread. The order she recommends is the right one: every time you correct an agent, turn the correction into a lint rule or an API change, clean up the old occurrences, strengthen verification, and only then automate the outer loop. It took her six months to get there, and she argues it will take others less.

Key Takeaways

Tan says she shipped 3,000 pull requests to production in a month, many of them while she slept, after six months of building her agents’ environment.
Her thesis: the job of a software engineer is now to maintain or improve code quality while massively increasing productivity, and that rests on trust, not blind trust.
The inverted trust pyramid puts deterministic layers (API design, architecture, lint rules, types, tests) at the base, verification in the middle, and prose rules like agents.md at the thin top.
Every correction to an agent should become a deterministic check or an architecture change rather than another line in a rules file.
The codebase is the agent’s largest input, so tech debt and workarounds get copied as if they were the established pattern.
Dune, which Tan describes as “kind of like Next.js for Electron apps,” encodes the team’s lessons as constraints and lint rules.
useEffect is banned in product code and allowed only inside Dune APIs, which removed a whole category of review comments.
Top-level directories for the Electron main and renderer processes, enforced by a dependency graph analyzer, stop agents importing main-thread code into the renderer.
About 99% of comments are banned because they go stale, humans skip them, and agents use them to justify shortcuts.
Runtime verification skills include a “feature map” of the app from the user’s point of view, giving agents and humans a shared vocabulary, plus reusable CLIs that save tokens.
The team uses TLA+ to model-check flows like notifications across devices, Lean to prove properties of a custom virtualization library, and the Bombadil library for property-based UI tests.
Grok Bot cuts a new stable release about once a day, and on each release an agent fuzzes the build like a real user, opens fix PRs for the bugs it finds, and auto-merges them.

Chapters

0:12 Turning Slop Into Gold

Tan opens by asking who has looked at someone else’s code and called it slop. Before agents, only outliers like Bret Taylor, who rewrote Google Maps in a weekend, were “coding machines.” Now anyone can be one, and the real skill is keeping quality high while output soars.

2:03 The Trust Pyramid and the Codebase as a Garden

She lays out the inverted pyramid of deterministic code, checks, verification and prose. Reviewing three colleague PRs a day in 2024 has become reviewing 300, so code arrives like a continuous organic stream. The constraints in the codebase are the concrete that new code grows around, and bad patterns spread like weeds.

5:00 Dune and Enforceable Architecture

Dune bakes desktop app lessons into constraints. Anything that needs nuance to use well tends to produce slop, so useEffect is banned in product code. Separate main and renderer directories, with imports between them forbidden, fixed the jank Cursor suffered when agents pulled main-thread code into the renderer.

6:56 Replacing Comments With Lint Rules

Agents over-rotate on small corrections and use markdown and comments to justify shortcuts. The timer workarounds at Cursor came with comments defending them, and the pattern spread. Grok Bot bans most comments and uses lint rules with helpful error messages instead, until eventually there is little left to review.

9:40 Runtime Verification and Feature Maps

Code review has shrunk into reviewing verification artifacts. Agents run the app themselves with tools like the Chrome DevTools Protocol, guided by a feature map so “the thingy in the left sidebar” has a name, and they reuse custom CLIs to save tokens.

10:50 Formal Verification and UI Testing

The team model-checks major user flows with TLA+ and uses Lean to prove its virtualization library is free of flicker and jitter. Both have already found bugs. Tan calls formal methods promising but not a silver bullet, one more layer alongside property-based UI testing with Bombadil.

11:51 Spending Tokens on Verification

Tan cites a developer who now spends 70% of her daily tokens on verification, and says her own usage looks the same. These are old techniques made affordable by agents. Her pstack plugin includes a state chart skill that visualizes state machines from the user’s point of view for review.

13:32 11,000 PRs and a 0.5% Revert Rate

The team shipped 11,000 PRs last month, releases a stable build about daily, and had about 227 contributors in October. Reverts held steady at about half a percent, and startup, typing and chat switching all got faster despite Grok Bot still being an Electron app.

15:11 Foundations Before Software Factories

Fix the codebase before automating: lint new anti-patterns, clean up old ones, and build a strong verification story. With those foundations, Grok Bot’s routines and projects can orchestrate hundreds of agents, including a release-time agent that fuzzes the app, opens fix PRs and auto-merges them.

Notable Quotes

“In many ways, the job of being a software engineer is kind of becoming like gardening.”
Lauren Tan, on code arriving as a continuous stream from agents

“Anything that requires nuance to use well often leads to slop.”
Lauren Tan, on why Dune bans useEffect in product code

“The worst part about comments though is that they’re not enforceable.”
Lauren Tan, on banning 99% of code comments

“Every time we correct our agents, it turns into one of these deterministic checks.”
Lauren Tan, on the team’s general principle

“We are entering a golden age of software testing and verification.”
Lauren Tan, on agents making rigorous testing economical

“If your inner coding loop produces slop, then automating that loop just produces more slop.”
Lauren Tan, on building foundations before software factories

Watch the full talk here.