Snow falls on the battlefield, settles on the ground, and units leave trails through it. Whether that snow looks better on the highest quality setting than on the one below it is not a question any test can answer. Somebody has to look at the screen and say which one is right.
The problem is that you cannot answer "is B better than A" by looking at A, changing a setting, waiting for a rebuild and then looking at B. That is not a comparison, it is a recollection, and recollections favour whichever one you saw second.
Starting a whole match to look at one thing
Checking a change to the snow used to mean starting a match. That brings up the map generator, two computer opponents, fog of war, pathfinding, audio, the full renderer and every one of the several dozen systems the game runs, and then you wait for a unit to walk somewhere snowy so you can look at the one thing you changed.
The cost is not only the waiting. A match brings its own variables: which landscape the generator picked, where the units happen to be, what the opponents decided to do while you were looking elsewhere, what the fog is hiding. Every one of those is noise against the single thing being judged, and none of them can be held still.
Isolation is also the only way to see a state the game rarely reaches. The comic-book outline around each unit breaks at the boundary between two detail levels, which in a real match is a moving line somewhere in the middle distance that you have to chase with the camera. On its own page it is a row of fixed stations, and you look straight at it.
A lab loads the thing under review and nothing else. There are twenty nine of these pages now, and the reason they keep getting built is not convenience. It is that most of the questions worth asking about how something looks are comparisons.
The rule that keeps a lab worth having is that it must load the real thing rather than approximate it. A preview that reimplements what it previews is a second implementation, free to drift, and the day it drifts is the day it starts lying to you with total confidence. The strictest version is the spell lab, which boots the same game the play page boots rather than a scene that resembles it, and what that buys and what it costs is a post of its own.
Stated generally: the preview imports what the shipped path imports, at the same point, with the same arguments. If the only way to build the page is to rebuild the thing it shows, you are not building somewhere to look, you are forking the feature, and the fork has no users to notice when it goes wrong.
Whether you need one: two questions
The test is short and it has nothing to do with how large the project is.
Does judging one change mean starting the whole application? Not "does it take a while", but "does reaching the state I want drag in decisions I did not make". A match here brings a random map, two opponents and a fog layer, and each is a variable moving underneath the thing being judged. The same is true of a checkout flow that needs a full basket, or a chart that needs a week of data in a particular shape. If getting to the thing costs you control over everything around it, the waiting is not the expense.
Are you comparing two states you can only hold one of at a time? Two quality settings, two easing curves, two colour ramps, the before and after of any visual change. All of those need both options in one frame, or the answer is unreliable.
One yes is enough to justify the page. Two noes mean a plain automated check is cheaper, and you should write that instead. The labs you can open from this post are the weather lab, the ink lab, the alphabet lab and the LOD lab, and each one below exists because a comparison was otherwise impossible.
Two quality settings in one frame, not two page loads
Open the weather lab and you get one canvas cut down the middle, snow or rain falling on both halves, a different quality setting on each side, and a control strip for choosing which two you are comparing. Same scene, same camera, same random seed.
The interesting part is what the two halves do not share. An earlier version used one weather system for both halves and re-ran its update after switching the setting, on the theory that every quality knob is read fresh each frame. The knobs are. The state behind them is not: one system owns one field of particles, one pool of contact rings on the ground and one smoothed intensity value. Re-running it for the second half either advanced the particles twice or, as it actually ended up doing, redrew the first half's particles at a different count. The halves were never independent. It was one storm drawn twice.
The fix is two separate instances kept apart by three.js layers, and it is short enough to lift into any comparison view.
// Two independent options in one canvas, sharing one camera.
const assignLayer = (root, layer) => root.traverse((o) => o.layers.set(layer));
assignLayer(optionA, 1); // e.g. weather at the highest setting
assignLayer(optionB, 2); // e.g. weather one setting down
// terrain, water, trees and clouds stay on layer 0, so both halves get them
renderer.setScissorTest(true);
for (const [half, layer] of [[0, 1], [1, 2]]) {
camera.layers.set(0); // see exactly layer 0 ...
camera.layers.enable(layer); // ... plus this half's own layer
const x = Math.floor((half * width) / 2);
renderer.setViewport(x, 0, Math.floor(width / 2), height);
renderer.setScissor(x, 0, Math.floor(width / 2), height);
renderer.render(scene, camera);
}
renderer.setScissorTest(false);One trap in that, checked against the real library rather than assumed: layers are not inherited. Setting a layer on the parent alone leaves every child on layer 0, where both halves can see it, and the symptom is a comparison that looks perfectly correct while showing you the same thing twice. Walking the whole tree is what makes it real, and it has to be re-run whenever anything adds a child.
Every detail level at every distance, including the ones the game would never pick
The ink lab lays out one station per distance, receding away from a real game camera at a real height and angle. Each station shows the full model, the animated flat card and the still flat card side by side, regardless of which one the game would actually choose at that distance. The one the game would pick is printed on the label rather than enforced.
That looks like the page showing something false, and it is deliberate, because both of the failures it exists to catch are comparisons. An outline drawn at a fixed thickness in screen pixels keeps that thickness on a unit filling 400 pixels of screen and on one thirty pixels tall, so the far one turns into a blob. Worse, the outline is produced by a different mechanism at each level, so a level missing its outline reads as a hard horizontal line across a tilted view rather than as anything to do with distance. Neither is visible in a frame containing one level. Both are obvious in a frame containing all three, which is why the outline post has a whole section on the boundaries between them.
A preview that cannot drift from the thing it previews
The alphabet lab draws nine faction alphabets of ten letters each, plus twelve magic school seals and the faction colour shift applied over all of it, on a plain 2D canvas. Not in 3D, on a page about a 3D game.
The letters ship as vector outlines and are drawn into textures at runtime by two functions. The lab calls those same two functions, at the same size, including the switch to a simpler version at small sizes. A 3D preview would have been a second renderer for the same outlines, and a second renderer is a second opinion. This one cannot disagree with the shipped art, because it is the shipped art drawing itself.
A miss has to look like a miss
One lab compares every version of a single model in a grid: detail levels across the columns, texture sizes down the rows, each cell reporting file size, triangle count and image dimensions. How it loads them is deliberately not how the game loads them. The game is handed a list in order of preference and walks it, so a missing small version silently resolves to a larger file and draws perfectly. In the lab, each cell asks for exactly one thing, fetches it, and reports what happened. A miss is a labelled empty cell, never a substitution.
That rule exists because a fallback had already hidden something. A whole set of simplified textures was completely empty and nothing anywhere reported it, because every request for the smaller version quietly served the larger one. Finding it took a page that refused to substitute, and it turned up 12,401 files that had been generated and never uploaded. A follow-up count of another set was first published as 681 MB and corrected to 711 MB, because the first figure came from a tool that reports mebibytes under a column labelled with an M. The wider version of that story is in the post about the download before a match.
True scale, or you are judging the wrong thing
The LOD lab draws a unit's detail levels at true in-game size. The game loads every unit at the same target height and then applies that unit's own scale to the whole group, model and flat card alike, so the only useful question is whether the card's character still matches the model's character at that size. Other model tools load things at five times scale for inspection, which is right for inspecting a model and useless for judging a size relationship.
The labs that are not linked from here, and why
Several of the twenty nine are safe to link in a post, and four of them are named above. The rest are not, for a mechanical reason rather than a secretive one: some of them call a paid image generation service, and some write files. A public link to a page that spends money when it loads is an invitation, so those are not named on this site at all and run only against a local development server. At least one refuses to do anything unless it can see it is running in development.
Everything on the list is marked as not for search engines regardless, and the two halves of that mechanism fail differently. A robots.txt entry asks a well-behaved crawler not to fetch a page, which does nothing about an address that gets indexed because something else linked to it. An X-Robots-Tag: noindex header tells a crawler that already fetched the page not to index it, and that is the layer that holds when a lab is linked from a blog post, which several deliberately are. One list feeds both the robots file and the header configuration, so the two cannot disagree.
What a lab cannot tell you
A lab proves a system works in a lab. The weather lab can show that the top setting buys something over the one below it, and it cannot tell you what either costs in a match with 800 units on an eight player map, because the whole point of the page is that none of those units exist. The bridge between the two is a separate benchmark with its own failure modes, covered in testing a game without a test framework.
So the division of labour is: a lab answers "which of these two is right", a benchmark answers "what does the right one cost", and a test answers "has it changed since". Building the first kind is cheap, and it is the only one of the three Adam can use directly, without reading a line of code, and still come back with a useful answer.
