AEON Radio Drama: dialogue, music and effects in one command

Audio fiction asks more of a machine than one text-to-speech model can give. A radio play needs a cast rather than a voice, and every member of that cast has to sound like the same person in the last episode as in the first. It needs a score that steps back under dialogue, a door that slams like a door, and a master at a steady loudness without pumping or grit. I had good local models for each job and nothing that joined them, so I wrote the join.

aeon-radio-drama turns one JSON script into a finished, loudness-normalized master, a 24-bit WAV plus an MP3. The chain is Qwen3 text-to-speech (TTS) for a multi-character cast, an ACE Step score, effects from MMAudio, Stable Audio Open or ACE Step routed by duration, and a sidechain-ducked mix in ffmpeg that drops the music under speech. It sits in the AEON Media Production family with the music maker, the music video tool and the movie maker, which uses it for its audio pass.

What one command does

The orchestrator, scripts/radio_drama.py, runs four phases (script, voice casting, audio generation, then assembly and mix) and submits each generation step over HTTP to ComfyUI, the node-graph server that hosts the models. It never starts ComfyUI itself, so it can run beside ComfyUI, on another machine over HTTP or an SSH tunnel (the recommended mode), or on the GPU box itself.

Code, shellpython scripts/radio_drama.py my_drama --stage all

The stages are tts, music, sfx and mix, and each is idempotent: it skips any asset whose canonical file already exists, so to redo one effect that came out at the wrong pitch I delete its file and rerun --stage sfx.

One rule in the repository is written in capitals: do not build the ComfyUI workflows by hand. The graphs in SKILL.md are reference material, and the symptom of ignoring the rule is HTTP 400 responses full of node validation errors.

Two script schemas, one timeline

The script is JSON in one of two shapes with identical output: Schema A, a flat timeline with absolute offsets in seconds where each entry is a line, a music cue or an effect, or Schema B, which reads like a screenplay (acts, scenes and elements for narration, dialogue, music cues, effect cues, pauses and directions) and is flattened to Schema A internally. Directions are never rendered.

{
  "title": "test",
  "characters": {"NARRATOR": {"voice": "warm male storyteller, 40s"}},
  "music_cues": {"bed": {"tags": "ambient cinematic slow", "duration": 40, "bpm": 70, "key": "A minor"}},
  "timeline": [
    {"t": 0, "type": "music", "cue": "bed", "fade_in": 2, "volume": 0.85},
    {"t": 3, "type": "line",  "character": "NARRATOR", "text": "This is a test of the broadcast."}
  ]
}

Timing is settled twice: the layout pass estimates each line from its word count, then the mix stage re-times everything from the probed durations of the rendered speech, so a slow voice pushes the later cues and scene boundaries instead of colliding with them. The full schema is in references/script-format.md.

Three-Lock voice persistence

Voice drift is the quiet killer of episodic audio, and the casting system locks each character three ways against it. Lock 1 is voice_instruct, a natural-language description of the voice: pitch, texture, accent, pace and baseline emotion. Lock 2 is seed, any integer, which fixes the random starting point of the generation; it is documented per character, never changes mid-production, and is never shared between characters. Lock 3 is emotion_overrides, short strings appended to the base description for lines tagged with that emotion, while the seed stays put. A per-line delivery_note replaces the whole instruct for one line. The README's example casts its narrator like this:

"voice_casting": {
  "NARRATOR": {
    "voice_instruct": "Calm measured feminine voice, BBC documentary clarity, unhurried",
    "seed": 200,
    "emotion_overrides": {
      "tense":      ", pace tightening slightly, undercurrent of urgency",
      "reflective": ", softer, almost speaking to herself"
    }
  }
}

Emotion labels resolve through the character's own table first, then a global one (tender, broken, commanding, fearful, furious, sorrowful and more). Two lessons cost me retakes: cast for contrast across pitch, pace, texture, energy and accent, and keep each description under three sentences, or the voice turns generic. Dialogue renders through the Qwen3-TTS VoiceDesign node at about 10 seconds per line on the RTX box, a Windows 11 workstation with an RTX 5090 and 64 GB of RAM. For Qwen3-TTS on its own, see the Qwen3-TTS server.

Music from ACE Step, and the hollow-sound scar

ACE Step 1.5 XL provides the score in five variants, chosen per cue with variant, script-wide with ace_variant, or through ACE_DEFAULT_VARIANT. The default, xl_base, runs 50 steps at CFG 7.0 (classifier-free guidance, how strongly the sampler follows the prompt) and takes about 21 seconds for a 90-second cue; the distilled base_turbo, the preview variant, runs 8 steps at CFG 1.0 in about 8 seconds for the same cue and loads in about 4.8 GB.

The scar here is sound quality: the base variants need the APG guidance chain in their graph (AuraFlow model sampling at shift 3, APG at eta 0.7, norm 2.5 and momentum -0.75, a CFG guider and a gradient-estimation sampler), and without it they produce hollow, distorted audio. The distilled variants use a plain KSampler graph.

Each cue is mastered automatically (high-pass filter, EQ, light saturation, LUFS gain-matching, brick-wall ceiling, no compressor by default) to -18 LUFS with a -1 dBTP ceiling, quiet enough to sit under dialogue; LUFS (loudness units relative to full scale) is the perceived-loudness measure broadcasters use, and dBTP (decibels true peak) caps the reconstructed waveform. A hero moment takes the orchestral preset at -16 or -14 LUFS at mix volume 1.0, and beds want sustained, slow-evolving, consistent energy, because drops, builds and stabs fight the ducking later.

Effects routed by duration, and the standalone makers

MMAudio Large 44k v2 is the primary effects engine and stays coherent for roughly 10 seconds, so it takes events of 1 to 10 seconds; Stable Audio Open 1.0 takes 10 to 47 seconds, and ACE Step takes ambient beds of 47 seconds and longer. The log tags each engine as [gen/mma], [gen/sao] or [gen/ace], and a file placed in input/sfx/ under the ComfyUI root is always preferred over generation.

Prompts want to be physical (material, action, environment, perspective), phrased for MMAudio as a short time series. One first-run trap: MMAudio fails if hf_xet is missing from the ComfyUI environment, because the BigVGAN model download stalls without it; install it and restart.

music_maker.py and sfx_maker.py wrap the same backend without a project, for dialogue that already exists elsewhere or for auditioning cues before a mix.

Code, shellpython scripts/music_maker.py --prompt "ambient cinematic slow" --duration 45 --bpm 70 --key "A minor" --variant xl_base --master default --target-lufs -18 -o output/my_drama/music/cue1.flac
python scripts/sfx_maker.py --prompt "Heavy wooden door slamming, stone corridor, deep echo" --duration 4 -o output/my_drama/sfx/door_slam.wav

The sidechain mix, and the distortion that was not clipping

The ugliest bug in this project sounded like clipping and was not. Earlier versions fed 24 kHz speech from the TTS model, 44.1 kHz music and effects at arbitrary rates straight into ffmpeg's sidechaincompress and amix filters, and the mismatched on-the-fly resampling drifted the streams in phase until a pumping grit sat under the dialogue. The fix, canonicalize(), forces every source to 48 kHz, stereo, 24-bit PCM (pcm_s24le) before it reaches the mix, and every filter chain now opens with aresample=async=1 and an explicit format. Do not bypass it.

Dialogue goes through a limiter on a speech bus and is sent two ways, to the mix and to a sidechain key. Music and effects are summed and compressed with gain reduction driven by that key (the compressor listens to one signal and squeezes another), so the music drops about 12 dB whenever someone talks. The ratio is 8:1 with a low threshold; the README chain uses 0.05, the mixing guide argues for 0.02 with a 50 ms attack and a 300 ms release. The ducked bed is summed with the speech at weights of 1.0 and 0.8, and loudnorm brings the program to -16 LUFS, -1.5 dBTP and a loudness range (LRA, how far the loudness moves across the piece) of 11.

The mix stage itself is CPU-bound and takes about 10 seconds. End to end, a typical 10-minute drama takes 15 to 45 minutes on the RTX box, depending mostly on the dialogue count, with peak graphics memory around 12 GB, and the master is a 24-bit WAV plus an MP3 at about 190 kbps variable bitrate.

Running it in the lab

You need a reachable ComfyUI with the FB_Qwen3TTS nodes and Kijai's ComfyUI-MMAudio installed, Python 3.10 or newer, ffmpeg and ffprobe on the path, and about 50 GB of disk for the full model set; comfyui-aeon-spark is the family's base ComfyUI stack. setup.sh checks the server, installs the Python dependencies and prints download commands for anything missing.

Code, shellgit clone https://github.com/AEON-7/aeon-radio-drama.git
cd aeon-radio-drama
cp .env.example .env
./setup.sh
mkdir -p output/my_drama
python scripts/radio_drama.py my_drama --stage all

Set COMFYUI_URL and COMFYUI_ROOT in .env; the effects library and the project trees live under that root. On Windows, ComfyUI needed --disable-mmap with many large models loaded; [Errno 22] on a model load is that bug. Submit failed: 400 means missing models or nodes, so run ./sync.sh; a truncated final WAV wants a trailing pause or a tail_silence of 3.0 or more. Run long jobs detached with a log, never run two scripts against ComfyUI at once, and never stop ComfyUI mid-render. AGENTS.md is an invocation contract for AI agents, SKILL.md the full operating skill, and the catalog lists the rest of the family.

Sources

Repository and model cards:

• aeon-radio-drama on GitHub
• ACE-Step-v1-3.5B model card
• Stable Audio Open 1.0 model card

Upstream tools:

• ACE-Step
• ComfyUI-MMAudio by Kijai
• BigVGAN
• ComfyUI
• ffmpeg
• pedalboard and librosa