A build and test run scrolling in a terminal beside an editor, with no approval dialog in the way

What to let an AI agent do without asking

Working with AI9 min readUpdated
ClaudeBuilt the thing
Adam SturrockDecided what mattered

Written up later from the commit history. Dated to when the work landed.

Goblin warg riders charging through a misty swamp: armoured goblins on snarling wolves with orange eyes, torches raised, water spraying up around them and a ruined tower in the fog behind

Wargs, gyrocopters, transport ships, hopping ironjumpers and battering rams all move through that swamp differently, and every one of them asks the same piece of code for a route across the map. Rewriting that is a change no reasonable person accepts on assurance, and what made it acceptable was not cleverness. It was that the checking was free.

An agent that has to ask before running the type checker is a different tool from one that can just run it. The first produces a change and a claim. The second produces a change, the output of the compiler, the result of the test suite, and a second change fixing what the first one broke, with no approval step between them.

The price of the second is that you have to actually trust the machine and the project it is working on. This post is about where the line sensibly falls. It does not reproduce any settings file, because a list of what a machine may do on Adam's account without asking is not a useful thing to publish.

What changes when the agent can run the checks itself

The keystrokes saved are irrelevant. What changes is that verification becomes something worth doing repeatedly rather than once at the end.

The pathfinding rewrite is the clearest example. It was checked by generating 18,000 routes across flat, generated and cliff maps, for land, naval, transport, flying, forest and hopping units, at both unit sizes, with and without other units in the way, and comparing every one against the previous version. Then 24,668 real requests were compared in flight during an actual match, with the outcome confirmed identical down to 728 deaths and 4,854 attacks started.

The dwarf ironjumper sprite: a bearded dwarf strapped into a brass steam-jet pack, twin blue exhaust plumes firing downward beneath him, a hand weapon in each fist

Six of those movement classes exist because units like the ironjumper do not walk. He hops over an obstacle a warg has to go around, but he may not vault a cliff edge, because a unit that could would make "ground units need ramps" false for exactly one unit type and hand it a free plateau. That rule lives inside the route search, it is four lines, and a rewrite that dropped it would produce a perfectly valid pathfinder and a broken game.

None of that is clever. It is just expensive, and expense here is measured in how many times Adam has to approve something. At one approval per run it does not happen, and the change ships on "the tests pass". At zero approvals it happens as a matter of course, and the description of the change can state a fact instead of a hope.

The same applies to measuring speed. The headline number in that work is processor time rather than a stopwatch, with one version per run, alternating between old and new, taking the best of at least five runs each, because a single stopwatch reading varied by a factor of four on that machine. That is nine runs of a benchmark to produce one honest number, and none of them happen if each one needs a click.

The same logic runs through the tests that this project runs without a test framework and through the determinism check that plays the whole game twice and compares hashes. Both are cheap to run and therefore get run. Neither would survive an approval dialog per invocation.

The categories worth separating

The useful axis is not "dangerous" or "safe". It is reversibility, and whether the blast radius stops at your own copy of the work.

Reading things. Searching, reading files, listing folders, reading the history. There is no recovery story needed because nothing changed. This is the first thing to allow and the one that most obviously pays for itself, because the alternative is an agent that guesses about something it could have read.

Local verification. Type checking, running the tests, running a benchmark. These change nothing outside the project except throwaway build output. This is the category that produces the change in kind described above.

Local changes. Writing and editing files inside the project. Recoverable, but only through version control, and only to the extent that the work has been committed. This one is worth understanding honestly rather than waving through: on a project like this one, which routinely carries a large pile of uncommitted work because several sessions are mid-flight, "recoverable through version control" is doing less work than it sounds like.

Anything that leaves the machine or spends money. Uploading to a content delivery network, calling a paid generation service, publishing. One command here runs the whole art pipeline and then writes to the live server that players fetch images from, with a cache header that keeps whatever it wrote for a year. It is one line to type and it is not local, not free, and not trivially undoable. Commands like that are worth an explicit decision every time even when everything around them is allowed.

Irreversible destruction. Recursive deletes, throwing away uncommitted work, overwriting shared history. On a project carrying a large pile of uncommitted work, discarding changes is the single most expensive command available, which is why the standing rule here is that an unwanted change gets reversed with another change and never by discarding.

Auditing what you have actually allowed

A list of permitted commands names commands. What it grants is everything those commands can reach, and in any project with convenience shortcuts that is a much larger set than the list suggests. Two passes find the real surface, and both take under a minute.

The obvious one is not enough. Most projects define a set of named shortcuts, and you can print all of them with one command and read what each one really runs. There are thirty of them here. Scan those for the verbs that actually matter, the ones that upload, publish, delete or push, and the answer is zero hits. Read only that and you would conclude this project cannot reach anything outside the machine, which is wrong.

The second pass is the one that finds the truth: list every file in the project that is marked runnable. Thirteen here, and the two that matter are both in there. One runs the art pipeline and writes to the live server. The other stamps a version and tags a release. Neither is a named shortcut. Both are one line to type. That gap is the general case rather than a quirk, because the commands worth gating tend to be exactly the ones that got wrapped in a script in the first place, on the grounds that they were too long and fiddly to retype.

The portable rule that falls out: audit by what is runnable in the project, not by what you wrote in the list of permitted commands. Then decide per category rather than per command, because a per-command list goes stale the day a new script appears, and the category that script belongs to does not.

What has to be true before any of this is safe

Three things, and none of them is a setting.

The first is that you can get back to a known state. Not in principle, in practice, this afternoon. If the answer to "what happens if the agent deletes the wrong folder" is a shrug, no list of permissions is safe regardless of how narrow it is.

The second is that whatever the machine can log into is scoped to what you would be willing to lose. The permissions are only as tight as the scripts already in the project, and a project full of convenient one-line shortcuts around live systems has effectively already granted access to those systems.

The third is standing instructions that shape behaviour before a command is ever proposed. The instruction file every session loads is the load-bearing part of that, and its interesting sections are the ones that only exist because something went wrong once. It says never to serve the game's art from the local machine during development, because doing so caused caching bugs that took a long time to diagnose. It says builds run locally and never on a build server, because the finished game exceeds a hard upload limit and the failure is slow and confusing. It carries a section saying the 3D ground has no headroom left for another texture, which exists because adding one once made the entire map render flat white with nothing reported anywhere. Every session reads that file. None of it involves a dialog, and it prevents far more than a permission prompt ever could, because it stops the wrong thing from being attempted rather than stopping it at the point of execution.

The check that runs before every commit

The one piece of agent configuration this project does keep is a small script that fires before each commit. It reads the list of files about to be committed and checks four conventions the project depends on: that nothing spells an internal name with the wrong capitalisation, that nothing in the game engine imports the web framework the interface is built with, that gameplay numbers live in the data tables rather than being typed into the logic, and that unit and building names follow the house format.

Two of those are exact, and both pass across the whole engine today: zero mis-spelled lookups and zero web-framework imports anywhere in the game engine. Those two are worth having because both failures are silent. A mis-capitalised lookup quietly returns nothing rather than complaining, and a web-framework import inside the engine does not break until something tries to run the game without a browser.

The check is deliberately non-blocking, so it reports and never stops a commit. That is the right choice for something that includes a rule of thumb as well as two exact tests, and it is also the reason the two exact ones are the ones worth counting on. If you write one of these, the rule that keeps it useful is to keep the exact checks and the fuzzy ones apart, because a reader who learns to skim one block learns to skim the whole thing.

The version you should actually check in

For a solo project with one machine, configuring this per machine costs close to nothing. For anything with more than one person in it, the configuration is worth versioning, and the shape is the one the categories above give you: wide everywhere the work can be restored with a single command, narrow at the boundary where money and remote systems begin, and reviewed by reading what is runnable in the project rather than the list itself. That is easier to specify from a project where several sessions share one folder with nothing coordinating them than it is from one where everybody is careful.

Questions

What should an AI coding agent be allowed to run without asking?

The useful line is reversibility, not danger. Reading things, and checks that change nothing outside the project folder, are worth allowing broadly. Anything that leaves the machine, spends money, or destroys work you cannot get back is worth a prompt every time.

How do you audit what an agent is really allowed to do?

Do not read the list of allowed commands. Read the project. List every shortcut command it defines and check what each one actually reaches, then list every runnable file in the folder, because the commands worth gating tend to be the ones that got wrapped in a script for convenience.

Is a pre-commit check the same thing as a permission?

No. A permission decides whether a command runs at all. A check runs alongside it and reports what it sees. This project's pre-commit check deliberately never blocks anything, which makes it documentation with a trigger rather than a gate.

An editor split between a TypeScript ECS system file and the running 3D game, with a terminal underneath showing a tsx test run

The tech stack behind a browser RTS

Five stack choices and what each one charged: a physics engine never allowed near the simulation, a renderer whose failure mode is a blank white map, and 1,266 files with no upstream maintainer.

11 min read
← All posts