Building an Autonomous Build Loop

I kept seeing the term “loop engineering” around: autonomous AI loops where you hand the model a whole spec and let it run. And I kept hearing about people vibecoding entire apps and projects like it’s a breeze. It felt like the next level of AI-driven development was unfolding before all of our eyes, and I wanted to know what all the fuss was about.

The premise is simple: you give an AI a “spec” and say you want it all implemented. Now, if you were to straight up instruct an AI to do this, it would undoubtedly fail. It’s about providing another harness layer to focus the effort into a tight loop and let it spin autonomously until it’s done. The autonomy sounded like the easy bit, a model will happily write code unattended for hours, and the write-ups were thin on the questions I actually cared about: how does the loop know when it’s finished, and what notices when it’s been building the wrong thing since lunch? The only way I was going to really understand it was to build one. It’s a Claude Code skill, and I called it ailoop.

Builders and a coordinator

There are really two jobs going on inside a loop like this, and most of the design came from treating them differently. Building (write this code, make this test pass) should be constrained and repeatable. Coordinating is everything else: picking the next ticket, deciding whether the last one is really finished, and the harder job of noticing when the plan itself has stopped making sense. Those are judgment calls, and when a loop fails it’s usually one of them that failed first, without anything obvious breaking.

Most of the loops I’ve seen hand both jobs to one agent. I kept them apart. A coordinator does the judging, on the strongest model I have access to, and never gets downgraded to save money. Disposable subagents do the building, on Sonnet, which is plenty when the ticket spells out what to do and something else checks the result.

Roughly:

spec ──▶ decompose into tickets ──▶ backlog
                                       │
                    ┌──────────────────┘
                    ▼
        pull the next ready ticket
                    │
                    ▼
        spawn a subagent to build it
                    │
                    ▼
        an independent check verifies it
                    │
              ┌─────┴─────┐
           passes       fails
              │            │
          mark done   diagnose, retry
              │
              ▼
        pull the next ticket

It all starts with decomposition. The spec gets broken into tickets, each small enough for one agent to finish in a sitting, and the tickets go into a dependency-ordered backlog. After that the loop barely reads the spec again; it works off the backlog, pulling whatever’s ready, building it, checking it, marking it done, then pulling the next one. Which ticket counts as “ready” is worked out by a small script from the dependency graph. I don’t want a model doing that kind of bookkeeping across a file that keeps growing; it will eventually misread it, and a wrong “ready” wastes a whole build.

Each ticket goes to a fresh subagent that has never seen any other part of the project. A cold agent can’t fall back on shared context, so the ticket has to carry everything: what to build, which files it may touch, which decisions are locked, how I’ll know it worked. Writing tickets to that standard is tedious, but it doubles as a filter: when I can’t write one that a stranger could complete cold, the ticket is too big or too vague, and it gets split before anything runs.

Checking the work

Builders report success whether or not they’ve earned it. There’s no malice in it. A model that’s a bit lazy or a bit lost will still cheerfully tell you everything passed. And if the loop takes those reports at face value, it has no error correction in it anywhere.

So a ticket only counts once something other than its builder has been over it. The type check, the existing test suite, a diff against the files it was allowed to touch, and then the slower question of whether it implemented the feature or just hardcoded whatever the test asserts.

The judging is the one place I don’t try to save money. Builders can be cheap because their mistakes get caught, but a judge that waves bad work through defeats the whole arrangement, so it stays on the expensive model even when the bill is annoying.

Building in parallel

Tickets that touch different files run at the same time, each builder in its own git worktree (an isolated checkout) so they can’t tread on each other. When they finish, the branches merge and every check runs again against the merged tree. Two tickets can pass separately and still be broken together, usually because one made an assumption the other violated, so it’s that second run on the combined tree that decides whether either ticket is done.

The oracle

The stopping condition took the most fussing of anything here. A loop with no definition of done either never stops, or stops when it feels finished, and I’ve seen enough confident wrong “all done!” messages to know what that feeling is worth. The usual answer is a script: exit 0 means done. A script only covers part of it, though. Build passes, server boots, endpoint returns a 200: scriptable, no problem. Whether the feature is actually implemented, rather than special-cased around the one input the test uses, is not scriptable; someone has to read the code. A checklist of judgment calls has the opposite problem, it can’t run itself.

So the definition of done, which I call the oracle, is a document holding both: runnable commands and prose criteria side by side, in a form an AI can execute and also reason about. A ticket passes when it clears the commands and survives the read.

Before any building starts, the loop goes over every check asking how a lazy builder might pass it without doing the work. “The function returns a value” is trivially gameable. “These three inputs produce these three specific, different outputs” is much harder to fake. Wherever there’s a cheap way through, the check gets sharpened before anything gets built against it.

Checks are sometimes wrong, too, and a loop won’t hesitate at a wrong check the way a person would. So the oracle can be amended, with rules about who amends what. A typo in a command, the coordinator fixes on its own. Anything that changes what counts as done comes to me, every time. And when a bug slips past a check and gets found later, the fix has to include tightening the check that missed it. The oracle starts out only as good as I was at writing it on day one; it needs a way to get stricter while the build runs.

The ledger

The loop runs in chunks. A few tickets, a stop so I can look things over, then a completely fresh context for the next chunk. Fresh because a long-running agent degrades as its context silts up with stale detail, and its decisions get worse before anything visibly fails. Resetting has an obvious cost, which is that the loop can’t rely on remembering anything, so whatever has to survive the gap gets written down.

That’s the ledger, an append-only journal of what the loop did and why. Decisions, retries and their causes, amendments to checks, the times it caught itself about to build the wrong thing. A new chunk starts by reading it. Between the backlog, the oracle, and the ledger, the whole state of the build lives in files rather than in any conversation.

What the spec has to contain

All of this leans on the spec, and a thin spec just doesn’t work. Every open question in a spec is a place where the loop either stops to ask me or guesses, and the guesses accumulate into drift, so a spec for ailoop is over-specified on purpose. At minimum it needs:

  • A build order. Riskiest thing first. The loop can invent its own ordering, but the spec author already reasoned one out, and the author’s is nearly always better.
  • Locked decisions. Stack, data model, the do-not-add-X list. A cold agent will re-litigate your framework choice on ticket forty unless the choice is marked closed.
  • An out-of-scope list. This is what lets it stop instead of gold-plating forever.
  • A checkable “done” for each phase. The one hard requirement. If done can’t be made runnable and readable, the loop refuses to start and asks me to supply it.

Still in the loop

I’m running a real project through ailoop as I write this, and I’m still in the loop myself: I write the spec and look things over between chunks, and when it hits something it can’t get past safely, it comes and gets me. I don’t particularly want to give any of that up. The relay work is gone, the “do this, now this” of passing my own plan to the model one message at a time, and I don’t miss it.

Whether any of this actually works, I can’t tell you yet. The project isn’t finished, so I don’t know if what comes out the other end is good, or if it got there any faster than I would have driving by hand. That’s the follow-up post: how the harness held up over a full build, which checks caught real problems, which ones just got in the way. So far, the effort has gone somewhere I didn’t expect. Hardly any of it was spent making the model write better code, since it was already good at that. The bulk went into verification, into knowing whether a ticket is really done and whether the checks themselves can be trusted. If I were starting over I’d write the oracle first and the loop last.