A terminal running a single test script, printing a column of ok lines and a pass count at the bottom

Testing a game without a test framework

Labs & Testing8 min readUpdated
ClaudeBuilt the thing
Adam SturrockDecided what mattered

The whole playable map rendered flat white. Not a crash, not an error message, just the ground gone: units and buildings still drawn correctly, standing on a featureless sheet. It shipped that way, because three.js reports a shader that fails to build by printing to the console and carrying on. Nothing crashed, nothing was recorded, and the only report was Adam looking at the screen and saying the map was white.

That divides the work. Adam judges the game by playing it, so anything visible is already covered by the best instrument available. What needs a script is everything he cannot see: a spell that quietly comes out as the wrong kind of magic, a benchmark measuring a smaller game than the one that ships, a shader that builds on one machine and fails on another. All three of those produce a game that runs perfectly well.

A test framework — Jest, Vitest, Mocha and the rest — supplies four things: a way to declare a test (describe, it), a library of assertions, a runner that finds and executes every test file without you naming them, and a combined report at the end. Most JavaScript projects treat all four as one indivisible decision.

There is none of that here. No Jest, no Vitest, no Mocha, and no runner in the dependencies at all. There are 248 test files, counting every file tracked in git across the whole repository whose name ends in .test.ts, .test.tsx or .test.js, and every one of them is a script you run directly. The rule is worth stating because the number moves with it: restrict the count to the game engine and the answer is 226, since the React components, the lab pages and the asset tools carry tests too.

What one of them looks like

Four parts, and none of them is clever: the command that runs it, written in the header so I do not have to guess; a six-line assertion helper; a printed tally; and a failure exit code so a caller can tell. Assembled, a complete test file is this and nothing else.

// Contract test for slug generation.
//
//     npx tsx src/lib/slugify.test.ts
 
import { slugify } from './slugify';
 
let passes = 0;
let failures = 0;
 
function check(condition: boolean, message: string): void {
  if (condition) {
    passes++;
  } else {
    failures++;
    console.error(`  FAIL: ${message}`);
  }
}
 
check(slugify('Ash and Iron') === 'ash-and-iron', 'spaces become hyphens');
check(slugify('  Ash  ') === 'ash', 'surrounding whitespace is trimmed');
check(slugify('Ash & Iron') === 'ash-iron', 'runs of punctuation collapse to one hyphen');
check(slugify('Ash!') === 'ash', 'a trailing hyphen is never emitted');
 
console.log(`\nslugify: ${passes} passed, ${failures} failed`);
if (failures > 0) process.exit(1);

Run it and it prints slugify: 4 passed, 0 failed and exits cleanly. Break one assertion and it prints the failing message, the tally reads 3 passed, 1 failed, and it exits with a failure. The only dependency is the thing that runs TypeScript directly, which is one line in the project's dependencies.

Two things about that file are worth stating as rules rather than as examples. The run line goes in the header because how to run it is the one piece of knowledge a reader cannot reconstruct from the code, and putting it anywhere else means it is not there when they need it. And the helper takes a message rather than working one out, because that message is the only description of intent the failure output will carry. "Spaces become hyphens" tells you what broke. "Expected 'ash-and-iron'" tells you only what happened.

The pattern is remarkably consistent for something no check enforces. Of the 248 files, 229 carry the run line in the header, 219 declare that same six-line helper, and 233 set an exit code. Three use Node's own built-in test wrapper, which is the only deviation in the whole set and reads like a different day's habit rather than a decision.

What this buys

A test here imports the real data and the real function and runs them. Not a mock, not a stand-in wired up by config, the actual thing under the project's own TypeScript settings. There is no transform layer between the test and the code, so there is no class of bug where the test passes because the mock was wrong.

That matters more than usual in a project where a lot of the truth lives in data tables rather than in logic. The spell test is not testing an algorithm. It is asserting that a spell called Hail of Stars does not come out as frost, because the word "hail" reaches exactly one spell in the game and that spell rains starfire, so the word is describing a trajectory rather than a school of magic. You cannot mock your way to that assertion. You have to load the real table.

It also removes an entire category of configuration. No test config, no transform map, no path-alias mapping to fight, no second TypeScript config for tests, and nothing to update when the web framework moves. A file runs or it does not.

The tests that could not exist in a normal runner

Two are worth describing because their shape is unusual.

The first counts textures in shader source. The 3D ground uses fifteen of them before one particular effect touches it, against a limit of sixteen that graphics cards are guaranteed to support. An addition once shipped needing three more, the shader failed to build, and the whole playable map rendered flat white with nothing raised and nothing in the crash reporter. So the test now builds the material, pulls the generated shader text back out, and counts texture declarations with a pattern match, asserting the addition adds at most one. It never opens a graphics context. It cannot, and it does not need to, because the failure is a blank screen and counting beats looking. The full version of that story is in the post about the texture limit.

A snow-covered citadel of black stone and gold spires under an aurora, mountains and a full moon behind it

The second guards against three lists drifting apart. The game's moving parts get registered in three places: the shipped game, the background thread that runs the simulation, and the stripped-down rig used for performance measurements. That last list had once fallen roughly twenty parts behind. It costs nothing and breaks nothing. It simply means the performance report describes a smaller game than the one people play, and three of those parts were then made faster against numbers that could not have come from that rig, because they were never in it.

The test now reads all three lists from source and asserts that everything in the shipped game is either in the measurement rig or on an exclusion list with a stated reason, that no exclusion names something the game no longer has, and that the ordering agrees, since order of execution is behaviour rather than speed. It even asserts each exclusion's reason is at least forty characters long, which is a crude proxy for "I thought about this" and a surprisingly effective one. That failure class is exactly what makes profiling believable or not, and the AI turning out to be 90% of the work depended on the rig measuring the whole thing.

The eleven that run on a name

Eleven of them have a name you can type without knowing where the file lives, and between them they cover the properties that must never break.

One replays three scenarios of 3,000 ticks each and compares a fingerprint of the world, which is what stands behind every multiplayer match. Another checks that every seat on every map size opens with the units and buildings it is supposed to have. Another is the drift guard above. Another generates ten dungeons across every difficulty and every theme and audits each one, including flooding outwards from where you start to confirm it can reach every camp, shop, pool, treasure, boss, altar and exit.

A cave passage lined with glowing mushrooms and standing runestones, a stream of blue light running down the middle

Then the performance family: one measures a run against a stored baseline, one compares two runs, and one drives six stress maps at up to eight factions on a large map, which is the shape of match that finds costs a two player game never will.

Dwarf shield lines meeting goblins and warg riders across a smoke-covered field at sunset, a burning fortress on the ridge behind

Buying discovery back in twenty lines

The one thing a runner gives you that this pattern does not is discovery: running every file without naming it. That is worth having, and it is not worth a framework, because a framework would also take away the thing that makes these files good, which is that they are ordinary scripts with no ceremony and no seam between the test and the code.

The convention is already there. The run lines are in the headers in a consistent format, the assertion helper is the same in 219 files, and 233 already exit with a failure code. So all that is needed is twenty lines that know the shape: walk the folder tree skipping dependencies, collect anything whose name ends in the test suffix, run each one as its own process with its output going straight to the terminal, and collect the exit codes. Print how many passed, list the ones that did not, and exit with a failure if any did. That gives up the combined coverage number and the watch mode a real runner provides, and buys the only property the pattern was otherwise missing.

If your test files agree on a shape, you do not need a framework to run them, you need twenty lines that know the shape. Pick the four parts first, keep them identical across every file, and the runner becomes a detail you can add on any afternoon rather than a dependency you build the project around.

The rest of what holds this project together is in the tech stack behind the game, and the one guarantee that is fully mechanical is the replay check, which has a documented procedure for making it fail on purpose. If you would rather see what the assertions are protecting than read about them, the fastest route is an eight player free-for-all in a skirmish, which is the same shape of match the stress runs drive.

Questions

Can you write tests in TypeScript without Jest or Vitest?

Yes. A test can be an ordinary script that imports what it checks, asserts with its own six-line helper, prints a tally and exits with a failure code. Run it directly and there is no config file, no transform step and no mocking layer. What you give up is the combined report, the watch mode, and running every file without naming it.

What do you give up by not using a test runner?

Discovery, the combined report and watch mode. You can buy discovery back with about twenty lines that walk the folder tree, run each file and collect the exit codes. Coverage reporting and watch mode are the parts genuinely worth a real runner.

How do you test a graphics shader without a graphics card?

Test the generated source rather than the picture. This project counts texture declarations in the shader text, because going one over the limit makes the shader fail to build at runtime and produces a blank surface with no error at all. Counting catches it in a script that never opens a graphics context.

← All posts