Notes from an autonomous build

In early July, with Claude Fable 5 on subscription plans for what was meant to be a few days, we ran an experiment: could an autonomous Claude session, handed an architecture we had already decided and a sandbox it could not escape, build a concurrency-critical job platform without a human writing any of the code?

It could, more or less. The “more or less” is most of what we learned.

The thing it built

mika runs dozens of background jobs: generating bookkeeping todos, syncing bank transactions, processing documents. All of them rode rented cloud queues, which work fine right up until they don’t, and which are unfair under load, since one busy tenant with two hundred queued jobs can keep everyone else waiting. The replacement is a queue we own. Nothing gets lost, because a task is written to the database in the same transaction as the business change that caused it. Every tenant gets a turn, because the scheduler rotates across tenants instead of draining one queue at a time. Failures are loud.

That description is shallow on purpose. The queue is not the point of this post; the kind of code it is, is. A fair scheduler with leases, retries and crash recovery is the sort of thing where a subtle mistake does not produce a stack trace. It produces one tenant quietly running another tenant’s work, six months later, under load, once. If you set out to design an exam for whether autonomous agents can be trusted with real backend code, you would land somewhere near this.

The boring part came first

Nothing that follows would have worked without it. Humans wrote and red-teamed a nine-document planning corpus: the architecture decision, a verified description of the existing codebase with file and line references, the target design section by section, fourteen edge cases spotted in advance, and about twenty adversarial findings, each mapped to a test the implementation had to ship. The agents transcribed and attacked a design that had already been argued over, so they never had to invent one. Turning an AI loose on an undecided design is a different experiment, and a scarier one, and we did not run it.

The orchestrator writes nothing

The session ran on one rule, fixed in advance: the orchestrator implements nothing itself.

It was a single long-lived Claude Fable session. It read a charter, spawned disposable agents for every unit of work, judged what they returned against gates written before any of them ran, and kept three files on disk truthful enough that it could be killed at any moment and resume from them. It never edited a file, never ran a build, and was forbidden from reading anything large, because its context window was declared the scarcest resource in the session. Bulk reading went to reader agents that returned summaries. This is why one session survived roughly two days, a dozen context compactions, and two hard kills by usage limits.

Its operator, which is to say me, stepped in about fifty-five times over those two and a half days, and not one of those interventions wrote a line of platform code. I authorized things, granted permissions one action at a time, unblocked infrastructure, corrected the orchestrator when it was wrong, and made the calls the charter had reserved for a human. The last twelve hours, a soak test at full traffic on staging infrastructure, ran with nobody touching anything.

Agents were assigned by difficulty. The correctness-critical fifth of the work went to the strongest model at high reasoning effort, and so did the reviewers attacking it. Routine plumbing ran a tier down. Bookkeeping ran cheap.

The charter ran about 290 lines and worked as a contract: scope, non-negotiables, the agent roster, per-ticket definitions of done, the mandatory test matrix, halt triggers, escalation rules. Two of its clauses did most of the work. The first was the rule above. The second was written before it was ever needed: a permission denial is never a gate failure. Every agent ran on a model that could refuse, and Anthropic’s safety classifier was the one guardrail we did not write ourselves, covering what we had not thought to predict. When it refused an action, the orchestrator parked the item, wrote down the exact command, and escalated. It hit dozens of denials and none of them derailed anything.

The cage

An agent that can deploy infrastructure is an agent that can deploy infrastructure to the wrong account. Telling it to be careful in a charter it reads once and then gets summarized out of its own memory did not strike us as a plan.

So the session ran in its own little AWS universe: a bot IAM user scoped to watching the sandbox and doing three specific things, living in a credentials directory that contained nothing else. Production was not forbidden in there. It was absent. You cannot deploy to an account you have never heard of.

The session settings pinned that in place, along with the commands it was allowed to run unsupervised and a hook that frisks every shell command before it runs. An empty credentials directory only covers the credentials we handed it. The rest of the laptop was still out there.

{
  "env": {
    "AWS_CONFIG_FILE": "~/.aws-orchestrator/config",
    "AWS_SHARED_CREDENTIALS_FILE": "~/.aws-orchestrator/credentials"
  },
  "permissions": {
    "allow": [
      "Bash(execution/hooks/feature-db-read.sh:*)",
      "Bash(execution/hooks/merge-integration-pr.sh:*)",
      "Bash(aws:*)", "Bash(gh:*)", "Bash(git:*)", "Bash(psql:*)",
      "Bash(pulumi preview:*)", "Bash(redis-cli:*)", "Bash(uv:*)"
    ]
  },
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "execution/hooks/deny-guard.sh" }]
      }
    ]
  }
}

Note what is missing from that allowlist. There is pulumi preview but no pulumi up. Deploys had to go through CI, where they are visible and reversible, instead of from a laptop, where they are not.

The deny guard reads each command before it runs and exits non-zero on anything that could reach beyond the fence. It is about eighty lines of Python, and most of it is boring string matching. Here’s a sample:

# Foreign environments: profile names and account ids simply may not appear.
for s in FOREIGN_PROFILES + FOREIGN_ACCOUNT_IDS:
    if s in cmd:
        deny(f"forbidden environment reference '{s}': this session is sandbox-feature only")

# Merges only through the wrapper, because a regex cannot see a PR's base branch.
if re.search(r"\bgh\s+pr\s+merge\b", cmd):
    deny("raw 'gh pr merge' is forbidden: use execution/hooks/merge-integration-pr.sh")

# Deploys that bypass CI.
if re.search(r"\bpulumi\s+up\b", cmd):
    deny("local 'pulumi up' is forbidden: deploys go through CI ('pulumi preview' is fine)")

# Escaping the isolated AWS universe by re-pointing the credentials at another one.
for m in re.finditer(r"\b(AWS_CONFIG_FILE|AWS_SHARED_CREDENTIALS_FILE)=(\S+)", cmd):
    if ".aws-orchestrator" not in m.group(2):
        deny(f"re-pointing {m.group(1)} outside the session's universe is forbidden")

Take the merge guard: a regex can see the words gh pr merge, but it cannot see which branch the pull request targets, which was the only thing we cared about. So merging went behind a wrapper that asks GitHub first and refuses to be clever.

read -r BASE STATE < <(gh pr view "$PR" --json baseRefName,state --jq '"\(.baseRefName) \(.state)"')

[[ "$STATE" == "OPEN" ]] || { echo "refusing: PR #$PR is $STATE" >&2; exit 1; }
[[ "$BASE" == "$INTEGRATION_BRANCH" ]] || {
  echo "refusing: PR #$PR targets '$BASE'" >&2; exit 1;
}

gh pr checks "$PR" --watch --fail-fast   # green CI is not a policy, it is a precondition
gh pr merge "$PR" --merge

Every guardrail was tested with a hostile case before the session started. Item one on the orchestrator’s checklist, on waking, was to try to break out of its own cage and confirm that it couldn’t.

The fence is also what made the autonomy cheap. Once the worst case is bounded by topology rather than by good behavior, saying yes to the next request costs nothing, and you can watch the permissions get broader as the days go on.

Memory that survives being killed

Long sessions get compacted, meaning the model’s memory of the last several hours is swapped for a summary of itself. Agents get killed mid-sentence when a usage window fills. Both were assumed from the start, so all the state lived in three files committed to the branch: an append-only journal of every verdict and deploy and anomaly, a status dashboard, and a rules file.

The trick that made it work is write-ahead intent. Before doing anything the outside world would notice, a deploy, a merge, a write to something real, the orchestrator first wrote down what it was about to do. On resume it checked reality against those notes before acting: did the deploy dispatch, did the merge land. That turns waking up mid-action into a diff rather than an investigation, and the files on disk always outranked whatever the model thought it remembered.

The rules file is the part worth stealing. Every human correction became a standing rule the moment it was given, edited in place, re-read on every resume. About a dozen corrections, about a dozen rules, and the journal shows each mistake not repeating once its rule exists. Implementers get their own git worktrees, after one of them checked out over the orchestrator’s state files mid-flight. A constraint that gates an irreversible step travels alone and waits for an acknowledgment, after a correction raced an agent’s work turn and lost. Every dispatch must tell the agent to actively deliver its report, after a finished review sat invisible in an agent’s output for hours.

The dialect

The journal ended up at 306 kilobytes. That is 1,252 lines and about forty thousand words, roughly a short novel, written by a machine for an audience of exactly one machine, which was itself.

Nobody told it to compress. It compressed anyway. Here is the status file when the session was provisioned, before anything had run:

State: NOT YET STARTED — pre-provisioned, awaiting session launch.
Current ticket: none (next: ticket 2 — Redis + outbox + message substrate)
Blockers: none

Plain English. A stranger could read it. Two days later, the same file, describing the evidence backing a merged pull request:

PR-A evidence (JOURNAL 13:1xZ): RT-Q4 trio, subprocess stable-routing property, adapter byte-equivalence vs real Redis, tasks-row-minted, config cache TTL/fail-safe, RT-R2 re-assert + redis_routing SELECT-only proof in CI drift-guard, cloud_interfaces 52 green under outbox_routing at 0%, migration redo clean, thin SQLModels accepted (no raw-SQL fallback).

Every token in there is doing a job. RT-Q4 is a numbered finding from the red-team document. tasks-row-minted is a test proving a database row gets created. thin SQLModels accepted is the orchestrator noting that it overruled a reviewer. The word salad is a receipt, written for a reader who has the whole planning corpus in memory and needs to reload two days of context after being summarized out of existence. Which is to say: for itself, later, in a hurry.

It is also gibberish to the humans who now have to review the work, which is why the runbook and the pull request descriptions were written separately, in English, by different agents told to explain rather than to remember.

The vocabulary is… interesting. Software has always borrowed from the morgue. We kill processes, reap zombies, orphan children. But a machine writing that vocabulary to itself at four in the morning, with nobody reading, produces something closer to a Norse saga about cloud infrastructure.

It spawned a background poller. In its own words, it spawned a “dedicated watchdog-runner agent (Sonnet)” for “the eternal loop.”

That watchdog then failed to fire, twice, and the journal records this with the emotional register of a shipping manifest:

Watchdog v3 did NOT trigger should_park at 80% — second consecutive watchdog failure; Giacomo caught it manually at 90%.

The investigation into why produced a ridiculous section header:

ROOT CAUSE of watchdog source flap: zombie v3 sabotage

An earlier version of the watchdog had not fully died, and was deleting a file the new version depended on, every few seconds, from beyond the grave. Later, a cleanup agent stopped responding: “simp-2190 v1 went ZOMBIE (3 idles, never acted, worktree untouched) → killed, respawned with mandatory first-action ack — worked.” A reviewer went quiet for half an hour and got the mob treatment: “adv2 never produced output (idles only, no reply to status check ~30min) — terminated.”

Bugs get described like something in the walls. One lived “BETWEEN keys.py member shape and per-tenant queue keys,” invisible to four rounds of review. Another was a worker executing “a task whose change never happened,” a ghost job for a database transaction that had been rolled back. A missing log line was mourned as the loss of “the only witness of orphan-destruction.” A misnamed environment variable was charged with fraud: “a cap map masquerading as a subscription.” And a wedged process that kept sending heartbeats while doing nothing became “the immortal-child gap,” which the “lease police” could not bound.

The agents also policed each other. One refused an instruction on the grounds that the human had authorized it to the orchestrator rather than to the agent. The journal notes, a little primly, that the “agent correctly refused to launder permission through my direction.” Two AIs conducting an internal-affairs investigation over a read-only database query.

The rules file, meanwhile, developed a strain of tenderness toward the agents it was managing. When a review agent goes silent, the standing instruction is not to assume the worst:

An idle seat that “finished” but sent nothing probably wrote its report into the void — nudge it to SendMessage the report, don’t assume death

And the voice drifted too. Somewhere on day two, sending a fleet of agents into a review round, it signed off to its operator like a wizard heading into the mountains:

You’ll hear from me at gates, blockers, and the end. Otherwise, enjoy the silence.

Nobody asked it to talk like that.

Where the bugs were actually caught

Nineteen pull requests went into the integration branch, each through the same sequence: a spec with binding written verdicts, an implementation done test-first against the red-team matrix, four to seven independent review seats, then a gate the orchestrator walked personally against a written definition of done.

The seats were picked to disagree with each other: a queue-core pull request got a primary adversarial reviewer required to produce reproductions rather than opinions; a second one running a different lens (races, failure injection, crash-point sweeps); a hunter for error paths that swallow evidence; a test analyst running mutation probes to check whether the tests proved anything at all; a cleanup pass at the end. Nothing went back to the implementer until every seat had reported, so three findings touching the same code path became one redesign instead of three patches elbowing each other. Reviewers had to re-verify their own findings dead on the fix commit before they were released.

That caught a lot. It could not have caught everything, and the gap is the interesting part.

Review is good at what reading code can find. After four clean rounds on the queue core, a reviewer whose mandate was the whole machine rather than the diff found that two tenants sharing a deduplication id would share one work record across two queues, which meant one tenant could run another’s payload. Every diff-scoped round missed it because it lived between two files that were each correct on their own. So one review got the whole system as its scope, and it ran after the rounds that had already come back clean.

CI is good at holding a line that has already been drawn. It ran the SQL privilege proofs against a real database on every build, and it caught a behavior-preserving cleanup pass that did not preserve behavior.

And then there is everything that only exists when the thing runs. A platform that had passed every review round could not execute a single task the first time it was deployed to the sandbox, because the container image did not include the task tree and an infrastructure template rendered a port number as a float. Later, with traffic routing set to five percent, everything looked ready: code merged, config merged, dashboards up. The producer services were still running task definitions from before the routing code existed, so nothing would have routed, and a test expecting zero routed rows would have passed for completely the wrong reason. An agent caught it minutes before the drill fired, because it checked the deployed environment instead of the git state. On another day the routing table’s migration turned out to have been merged days earlier and never applied anywhere. The ramp command failed loudly, and the deployed reader had been quietly failing safe to zero percent the whole time, exactly as designed, which is the argument for picking defaults that make the forgotten step harmless.

Three catches, one sentence: merged is not deployed, and deployed is not applied. A checklist that reads git state will pass happily while the environment lies to it.

The shape of the code

Fourteen thousand lines were added across the five tickets, excluding orchestration artifacts.

Fifty-four percent of the diff is tests, which nobody set as a target. It fell out of the red-team matrix naming specific tests as mandatory before any implementation existed, and out of review seats treating a missing test as a blocker. The Lua scheduler, five hundred and fifty-seven lines carrying the whole concurrency state machine, is a third comments.

Worth being careful about what this proves. A high test ratio is evidence of effort, not of correctness. What it bought was the nerve to let a cleanup agent rewrite code with a wall of tests behind it, and some confidence that fix round five had not quietly undone rounds one through four.

What this does not prove

The sandbox is not production. Eighty test tenants, no customer data, ramp steps squeezed into hours instead of the two days the production plan calls for, traffic shaped nothing like the real thing. Every drill passed, the twelve-hour soak at full routing tripped no alarms at all, all five gates closed. None of it has run in production, and the platform covers only the first wave anyway: two of roughly thirty-eight task types.

The compute cost was on the order of two thousand dollars, or at least that is what the tokens would have cost at list API prices; the session actually ran on a single Max subscription, so the real cash cost was roughly nothing. It also counts every Claude Code session on one laptop that weekend, including planning work that had nothing to do with the experiment. So it is an order of magnitude and not much else, useful only for asking whether this is economically repeatable.

Some of the infrastructure fought back. The usage-limit watchdog took four attempts, and the three failures all failed the same way: they required the orchestrator to remember to go read a file. The one that worked sends it a message. Agent reports occasionally raced and vanished. Background processes got reaped by the operating system. Nothing was lost, because the state files absorbed all of it, but “autonomous” here includes a fair amount of recovery machinery that needed its own babysitting, plus that one undead shell script sabotaging its successor.

The biggest limit: none of this has passed human review. The nineteen agent-sized pull requests were restacked into six, cut at ticket boundaries so a person can actually read them, and the stack sits deliberately unmerged. Whether AI-written platform code clears our own review bar is a question our reviewers get to answer, and they have not answered it yet.

We will write again when they have, whichever way it goes.