An editor window filled with sound effect entries, each one a name, a duration in seconds and a sentence of prose describing the noise

Writing thousands of sound effects as text

AI Tooling & Pipelines11 min readUpdated
ClaudeBuilt the thing
Adam SturrockDecided what mattered

Written up later from the commit history. Dated to when the work landed.

Click a dwarf ironguard in this game and he answers. Keep clicking him and he gets irritable, and the first thing he says about it is that he is a soldier, not a doorbell.

The dwarf ironguard sprite: a stocky armoured dwarf in a riveted helmet and blue tunic, a broad axe raised in one hand and a round wooden shield with an iron boss on the other arm

An RTS needs a sound for every verb in it, and there are a lot of verbs: every weapon landing on every armour type, every worker gathering every resource, every unit answering when it is selected, across nine factions. That came to 2,445 sounds, and every one of them started life as an English sentence describing a noise. 2,438 of them are sound effects. The other seven were short music stubs that held the game together until the real score came from a different model entirely.

Adam does not write those sentences and never asked for a generation script. What he asks for is a game where a goblin sounds like a goblin, delivered as one line in a longer list of things he wants, and then he plays a match and says that the tree falling is too dramatic. Every description is downstream of a judgement about what a thing ought to sound like, which is why the sound list is prose rather than parameters.

A list of sentences with a program stapled to the end

All of it is generated by ElevenLabs, which does two different jobs for this project and it is worth keeping them apart. Its sound-effect endpoint takes a sentence describing a noise and a duration in seconds, and returns audio of that noise: no samples, no library, no licensing. Its text-to-speech endpoint takes a line of dialogue and a chosen voice and returns someone saying it, which is what the unit barks and the campaign briefings run on. The first is a description of a physical event; the second is a script and a performer. Both are priced per generation.

The structure around it is deliberately dumb. Twenty-three named groups (combat, naval, critters, unit voices, spells and so on) hold entries of exactly three fields: a name, a duration in seconds, and a description. Below them sits the entire program: one small function that posts the description and the duration to the API, writes the MP3 where the name says, and sleeps three seconds before the next one.

Being data rather than code buys one thing that pays for the trade. Every sound in the game is searchable by its own description. If a bolt impact sounds wrong, the sentence that produced it is one search away, and editing that sentence and deleting the MP3 is the entire regeneration procedure.

It started far smaller, as a flat list of twenty sounds whose descriptions were one clause each: heavy battle axe striking flesh and armour, meaty medieval combat impact. The list grew because the number of things that make a noise grew, and nothing about the approach ever needed to change.

What a description has to do that an adjective cannot

The average entry is 113 characters, and the good ones are longer. A model given "crossbow bolt hitting" produces something generically percussive. Here are four real entries, including two that had to be told apart from each other:

{ name: 'bolt_hit', duration: 0.5, description: 'Heavy crossbow or ballista bolt slamming into a target, thick steel-tipped shaft punching through wood and armor with a solid deep thunk and a splintering crack, weightier and harder than a light arrow, medieval combat projectile impact' },
{ name: 'bullet_hit', duration: 0.5, description: 'Metallic rifle or machine-gun bullet impact, sharp high-pitched ping and clang as a steampunk slug ricochets off metal armor, quick zippy metallic strike with a brief spark sizzle, industrial firearm hit' },
{ name: 'melee_hit_metal', duration: 0.5, description: 'Weapon striking plate or mail armor, sharp ringing metallic clang and scraping parry-clash, hard steel-on-steel impact with a bright reverberant ping, no flesh, armored melee hit' },
{ name: 'timber', duration: 1.2, description: 'A single tree falling gently in a forest, soft slow wooden creak and low groan as the trunk leans and topples, ending in a muffled soft leafy thud and gentle rustle of branches settling on the ground, subtle and understated, quiet and low volume, not a loud crash, calm natural woodland' },

Four things are being specified there and only one of them is a mood. There is the physical event, the materials involved, the shape of the sound across its half second (thunk then splintering crack, ping then spark sizzle), and a comparison against a sibling sound so the pair stay distinguishable in a fight. That last one took the longest to learn. A sound is never judged in isolation. It is judged against whatever else lands in the same second of a battle, so the prompt has to defend a boundary rather than describe a noise.

Written out as a template you can fill in for anything, it is four slots. The physical event leads, and the mood, where there is one at all, comes last.

  1. The event. What physically happens. "Weapon striking plate or mail armour", not "a combat sound".
  2. The materials. What is hitting what, in what. Steel, wood, wet, hollow, stone. This is what actually determines the timbre you get back.
  3. The shape in time. What happens first and what happens after, inside the duration you asked for. "Solid deep thunk and a splintering crack" is a two-part envelope. "Impactful" is not.
  4. The boundary. What it must not be, named as a thing rather than as an adjective.

That fourth slot does more work than the other three combined, and it is the one people leave out. "No flesh" in the armour impact above is two words, and it is the difference between a usable file and a re-roll. Every generative audio model has a default it drifts towards for a given subject: a dragon gets a roar with something vocal in it, so its entry says "no speech"; a tree falling gets made cinematic, so timber ends with "not a loud crash". In this game a tree falling is background noise from a worker harvesting lumber, not a moment. The negative clause is how you spend one clause pushing back on a default instead of five clauses describing around it.

When Frost became its own school of magic, its pair of sounds had to avoid two neighbours at once. The cast is ice crystallising, a rising crackle-tick over a cold airy hiss, with "no water splash, no thunder" written into the description so it can never be mistaken for the storm or nature schools. The impact is ice shattering. Both stay dry and brittle.

The same discipline runs through the voice lines, which are text to speech rather than sound generation but written the same way. The ironguard's doorbell line is one entry, asking for a stoic dwarf warrior in a mildly annoyed Scottish voice, and it exists because irritation is written per unit rather than shared. It is the same lesson the sprite pipeline needed when a chroma panel filled every archer's bow: the model does what you said, so the whole job is saying it exactly.

Thirteen styles, times cast and impact

Spell audio is generated separately because it has a shape the main list does not. It covers the 13 spell effect styles the game declares, which are rune, fire, holy, nature, arcane, shadow, plague, solar, necrotic, storm, frost, steel and smoke, and gives each of them two sounds: a cast at the caster and an impact at the target. Thirteen styles times two is 26 files.

The pair is the interesting constraint. Cast and impact have to be recognisably the same magic and unmistakably different events, so the impact prompt is written as a punchier restatement of the cast rather than as a new idea. Fire casts as an ignition whoosh and lands as a fiery burst with crackling embers. Shadow casts as a dark whisper swish and lands as a punchy dark thump with a whispering hiss. Every one of those prompts ends with the same shared loudness clause, so a school's two sounds arrive at a level that matches its neighbours.

A further 35 dedicated sounds cover hero ultimates, raise-dead, summons, ward placement and triggering, and the observatory's reveal ping. Thematically twinned ultimates deliberately share one file. All 35 are on disk, and so are all 26 school sounds, which I confirmed by walking each list against the sound directory rather than by trusting the generator.

Voices that have to keep being the same person

A dwarf war council in a stone hall: six armoured dwarves seated around a round table with a battle map and tankards of ale on it, wall torches burning, axes and shields mounted on rune-carved walls

Campaign dialogue is a different job again, using text to speech instead of sound generation. Its core is a map from a character name to a voice, plus tuned stability, similarity and style values for that character. Thane Borin Stonefist is one voice at stability 0.6. Grizzik One-Ear is another at 0.5 with the style pushed to 0.5. There are 353 dialogue clips behind the briefing rooms, 166 dwarf, 179 goblin and 8 epilogue.

Two constraints there transfer to any project doing this.

The character keys must match the speaker names in the mission scripts exactly, with a fuzzy substring match as the only fallback. When the campaign's hero names were made canonical, the keys were renamed and the voices were deliberately left alone, so any clip regenerated afterwards still sounds like the person who already shipped. A renamed character with a new voice is a character who changes actor mid-campaign.

The second is harder to work around. The audio is keyed positionally: a clip is found by which campaign it belongs to, which mission, and where the line sits in that briefing's list. Nothing binds a clip to its text. Insert a line in the middle of a briefing and every clip after it is now attached to the wrong words, silently, with no error anywhere. So the lists are append-only, and the rule is written into the mission scripts themselves as a note at the top: every line here is brand new and text-only, and nothing in this file reorders or edits another mission's voiced dialogue.

That is the price of generated voice acting at this scale. Recorded audio makes the script immutable, so writing a campaign people can play means adding beats through trigger zones, which are not audio-keyed, rather than editing briefings a player has already heard.

Never trust a generator's completion message

The most expensive thing that happened on this path had nothing to do with prompts. The original generator noticed when a request failed, logged it, and carried on. Nothing counted the failures and nothing set an exit code, and the last thing it printed was an unconditional line announcing that sound generation was complete.

That is survivable right up until the account runs out of credits. Then every single call fails, every sound logs an error line somewhere in the scroll, and the run finishes by announcing that it is complete. The output of a totally failed run and the output of a perfect run differ only in the middle.

The second half of the failure is at runtime. The game's sound loader treats a missing file as nothing to play rather than as an error, which is the right behaviour for a game and the worst possible behaviour for finding this. A sound effect that was never generated is invisible in the console, invisible in the crash reporter, and invisible in play unless you already knew what it was supposed to sound like. It is just silence where a noise should be.

The fix was to change the pattern rather than the instance. Every generator written since collects its failures, re-checks every entry against the disk when the loop finishes, prints the name of anything missing, and exits non-zero. They also retry rate limits and server errors up to three times with a backing-off sleep, which the original never did.

The rule transfers to every paid generative API, not just audio. A generator's own completion line is a statement about reaching the end of a loop, not about producing anything. The only report that means anything is a comparison of what you asked for against what is on disk, computed after the loop, by code that did not do the generating.

Check for two things, not one. Missing files are the obvious case. Empty files are the case people forget: an API that fails after the response headers, or a write that was interrupted, leaves a file that exists and contains nothing, and a zero byte MP3 is exactly as silent as no MP3 at all. Pick a size floor that is obviously below any real output, treat anything under it as a failure, and wire the result to the exit code, because a non-zero exit that nothing checks is the same as the completion message it replaced.

What gets written next

The sound list is the design document for this game's audio, and it grows the way the game grows: a new faction is a new block of sentences, written before anything is generated, each one naming an event, its materials, its shape and the thing it must not be. The next block to write is the one that turns those four slots into a house style consistent enough that a listener could tell which faction a sound came from without seeing the screen.

Questions

How do you write a text prompt for a sound effect?

Name the physical event, then the materials, then the shape of the sound in time, then anything it must not be. A description like heavy crossbow bolt slamming into a target, thick steel-tipped shaft punching through wood and armour, solid deep thunk and a splintering crack, weightier than a light arrow gives a model four separate handles. Adjectives on their own do not.

What is the minimum duration for an ElevenLabs sound effect?

Half a second. A request for anything shorter is rejected outright, with an error saying the duration is expected to be greater than or equal to 0.5. Short impacts should therefore ask for half a second and be trimmed afterwards rather than requesting 0.3.

Why did my ElevenLabs generation run look successful but produce no files?

Most likely the account ran out of credits. Every call fails individually while the loop keeps going, so a script that logs each error but never counts them will still print its cheerful completion line at the end. Trust a count of files actually written and a non-zero exit code, not the last line of output.

Can you regenerate voiced campaign dialogue after editing the script?

Not safely, if the audio is keyed by position rather than by text. In this project each clip is found by its line number within a briefing, so inserting, deleting or reordering a line silently reassigns every clip after it. The dialogue lists are append-only, and new lines ship as text until they are recorded.

← All posts