< Don't Run It Twice

Watch the CI on one of our agent PRs and most of what it does already happened ten minutes earlier. The agent that wrote the change ran tsc, lint, ran the unit tests, and only then pushed. It has to as I've got in a hook. CI then did all three again from scratch, and took ten minutes over it.

With most of our commits written by agents[1], that is a lot of repetition. The agent already ran the checks, and a hook can (mostly reasonably) record that.

So that's what we built: The agent leaves a git trailer under the commit message[2] recording what it ran before it pushed: tsc, lint, the unit tests.

The PR build reads the trailer and skips just those steps.

Over the last few weeks our builds have gone from ~10 minutes to 1.5 minutes. A good third of that was this change. The wall-clock saving was the obvious win, but the bigger one was contention: fewer Buildkite agents (the "runner" kind of agent)[3] get spun up, fewer jobs sit waiting for one, and the ones that stay warm cache better. At 100+ PRs a day that is a lot of build minutes that can be banked.

The obvious question is trust, since a trailer is just a claim. Anything that will go to main or an environment is forced to run all the steps again. This step stops all the churn on the all the intermediate branches and pushes.

This is a poor man's Bazel. Bazel does it properly, keying every action on a content hash and pulling the result from a remote cache[4]; this is keyed on a hook's record of what ran. It works because the agent did the work, the hook saw it, and the thing that matters gets checked again on main anyway.

Never Pick It Up Twice was about not touching a PR twice, and this is the same rule for the machines.

Also read: Never Pick It Up Twice, The Importance of Backpressure, and What does it take to build towards 100 PRs/day per engineer?.

Discuss and share via the meta page . Filed under AI, Code, and 100PR.

Footnotes

  1. On a busy day that is somewhere between 40 and 100 merged PRs, most of them agent-authored, which is what Building Towards 100 PRs a Day and One Hundred, and Fifty are about. ↩

  2. Git trailers are the Key: value lines at the end of a commit message, Signed-off-by being the one everybody has seen, and git will parse them for you. ↩

  3. The Buildkite agent is the build runner that polls for work and runs a job, so every step you skip is an agent somebody else's PR can have instead. ↩

  4. Bazel's remote cache maps a hash of each action's inputs and command line to its outputs, so a reproducible step never runs twice anywhere on the team. ↩