AEON Movie Maker: cinematic video with LTX 2.3 22B

By the time the audio side of the lab worked, I had dialogue masters from aeon-radio-drama and scores from aeon-music-maker, and nothing to put on screen. Lightricks' LTX 2.3, a 22-billion-parameter video model, changed the arithmetic: its distilled fp8 checkpoint, eight-bit weights of a model trained to reach a good frame in fewer denoising steps, renders a second of output in about 3 to 5 seconds of wall time on the Spark in fast mode, which the README puts at roughly 10 to 15 times faster than WAN-based pipelines.

AEON Movie Maker is the Python command-line client I built around it. It assembles ComfyUI workflows and posts them over HTTP: clip renders one shot, screenplay renders a multi-scene film from a JSON file, stitch muxes dialogue, music and effects over the clips with sidechain ducking, and concat-relay joins Prompt Relay sequences, all under the MIT license.

What the stack needs

ComfyUI must be reachable at COMFYUI_URL; in the lab it is the comfyui-aeon-spark stack. The client needs Python 3.10 or newer with requests as its only dependency, ffmpeg and ffprobe, and about 80 GB of disk for the fast and quality fp8 checkpoints, the always-on LoRAs and the Gemma 3 12B text encoder. A LoRA, a low-rank adapter, is a small add-on weight set that steers a model without retraining. Fast mode needs about 20 GB of VRAM with the models warm.

Code, shellgit clone https://github.com/AEON-7/aeon-movie-maker.git
cd aeon-movie-maker
cp .env.example .env
./setup.sh
python scripts/movie_maker_fast.py clip --prompt "drone shot over a misty pine forest at dawn" --duration 5 --width 832 --height 480 --output forest_drone.mp4

On the RTX box, an RTX 5090, a 7-second clip at 832x480 takes about 75 seconds warm.

Fast mode runs the distilled checkpoint with the IC-union and VBVR LoRAs at euler, CFG 3.0 and 20 steps; CFG, classifier-free guidance, is how hard each denoising step leans on the prompt. Quality mode runs the non-distilled dev fp8 with the distill LoRA at 0.5 and 30 steps, 30 to 50 percent slower; fast omits that LoRA because distillation is baked into its checkpoint. Abstract mode drops the always-on LoRAs and uses euler_ancestral at CFG 5.0 and 30 steps, because VBVR enforces gravity and object permanence, wrong for fractals.

Two levers of continuity

A screenplay is JSON: a characters dict of names to visual descriptions, an optional top-level negative_prompt, and scenes with an id, a duration, a prompt, the characters present and optional dialogue.

{
  "negative_prompt": "music, soundtrack, score, instruments, drums, melody",
  "characters": {
    "ALICE": {"description": "young woman, dark hair, blue eyes"},
    "BOB": {"description": "older man, gray beard, leather jacket"}
  },
  "scenes": [
    {"id": "01_doorway", "duration": 5, "prompt": "Alice in a doorway, watching the rain", "characters": ["ALICE"]},
    {"id": "02_line", "duration": 8, "prompt": "close-up on Alice as she speaks, then turns away", "characters": ["ALICE"], "dialogue": [{"character": "ALICE", "text": "I never said I'd stay forever."}]},
    {"id": "03_street", "duration": 6, "prompt": "Bob crosses the wet street", "characters": ["BOB"], "relay_break": true}
  ]
}

Each scene becomes one silent clip, and scenes over 7 seconds are chunked, because coherence degrades past about 8 seconds. The first lever of continuity is last-frame carry-forward, on by default: ffmpeg grabs a frame 0.3 seconds before the end of each clip, stages it under the ComfyUI input directory, and the next chunk starts from it as image-to-video, I2V. SKILL.md calls it the single biggest lever for coherence; --no-carry-last-frame turns it off. How tightly the next clip holds that frame is --persistence, 0 to 1, mapped inversely onto the strength of the in-place I2V node: unset is 1.00, 0.3 is 0.82, 0.5 is 0.70, 0.7 is 0.58 and 1.0 is 0.40.

The second lever is the seed. character_seed_offset(name, base_seed) is a stable hash, so the same name with the same base seed gives the same face from first scene to last. A scene's focal character is its first speaker who is not the narrator, else the first name in characters; its seed is the base seed plus a thousand times the scene index, shared by all of its chunks. For a series, copy the characters dict and the base seed between episodes.

Prompt Relay: dialogue and lipsync in one pass

With --use-relay, the screenplay runs through the PromptRelayEncodeTimeline node, which morphs through a prompt timeline in one forward pass and produces joint audio and video, with lipsynced dialogue, from lines written as CHARACTER says "line". One pass is capped at 489 frames, about 20.4 seconds at 24 fps and 640x640 on the Spark, so the film is cut into sequences: a new one begins when the frame budget would overflow, when a scene sets relay_break: true, or on a transition, cut or scene_change tag, and the last frame of each sequence is uploaded as the next seed. The wrapper prompt, built from style, setting and the character descriptions, is the strongest identity signal across sequences. On the Spark, per-scene rendering costs about 5 to 8 seconds of wall time per output second and relay 6 to 8, plus about 30 seconds of startup.

Speech is where the scars are. LTX cuts a line mid-word at the segment boundary and nothing downstream repairs it, so a scene gets at least its word count plus two seconds, never under five, with one exchange per scene and visible action after the line. In dialogue-free sequences the model sometimes narrates on its own, atmosphere or gibberish: transcribe each sequence with qwen3-asr-server, mute the bad ones with volume=0 and re-run concat-relay, always on the per-sequence files, after confirming real silence with an RMS of minus infinity, since silence can transcribe as a stray "Hoy."

LoRA strengths and the saturation guide

Scene tags route extra LoRAs, at most three beyond the always-on pair, each at its own strength: transition and pose at 1.0; zoomout and galaxy at 0.9; dolly-left, jib-down, talkinghead, claymation and cyberpunk at 0.8; ghibli, tribal and illustration at 0.7; ghibli_offset at 0.6. The seven Civitai style LoRAs need a token, and three or more together distort. The docs disagree on the always-on default, 1.0 or 0.7, so check the script.

The saturation guide exists because of one easy mistake. An older LTXV 2B distilled fp8 checkpoint, about 6 GB, loads and renders saturated, distorted frames; the right file is the 22B distilled fp8, so go by the file name, not the size. Check the checkpoint at its exact path first, then the text encoder, which must be the .safetensors Gemma file, since a .gguf encoder gives incoherent video. Then lower the LoRA strengths, starting both VBVR and IC-union at 0.5. Then adjust CFG and steps: in fast mode raise CFG to 4.0 if motion is loose and lower it to 2.5 if frames look burnt, in quality mode go up to 4.5, and never past 6, where every LTX 2.3 variant goes saturated, noisy and posterized. Then check the VAE, the autoencoder that turns latents into pixels; the wrong one shifts colours. Last, disable the style LoRAs, add them back one at a time, and warm the cache with a throwaway 3-second clip.

The final cut

stitch is the per-scene flow's mux. The dialogue master, from aeon-radio-drama with voices from qwen3-tts-server, goes to a speech bus and a limiter; music and effects pass through a sidechain compressor keyed by the speech, which turns the bed down whenever dialogue is present, at threshold 0.05 and ratio 8, so the music sits about 12 dB under speech, and loudnorm finishes at I=-16, TP=-1.5, LRA=11.

Relay films follow Recipe D in AGENTS.md, because the model adds a music bed of its own that fights any score. Suppress it with a top-level negative_prompt that names music, soundtrack, score, instruments, drums and melody. Join sequences with concat-relay --xfade 0; any crossfade blends the audio too and clips dialogue at the edges. Compose 5 to 10 cues in aeon-music-maker at the story's pivots, each the length of its zone plus a 2-second tail, joined with acrossfade at d=2. Force 48 kHz throughout and never use loudnorm linear=true, which leaves the audio shorter than the video. Dialogue is levelled at the source: production runs spread up to 23 LU between sequences, from -10 to -33 LUFS, so each sequence gets its own volume=NdB gain toward a -23 dB mean under a -1.5 dBTP ceiling. No sidechaincompress here: the score sits at 0.25, about -12 dBFS under speech and never below 0.18, and the final amix runs with normalize=0.

The remote-mode constraint

Screenplay mode and any I2V clip write seed PNGs into input/_movie_fast_frames/ under COMFYUI_ROOT, so the CLI needs filesystem write access to a path the ComfyUI server can also read. Pure-HTTP remote mode does not work for I2V or screenplays, only for single text-to-video clips. Three setups work: everything on one machine; the CLI on the GPU machine over SSH, which is my recommendation for screenplay work; or the remote input folder mounted over NFS, SMB or sshfs, with COMFYUI_ROOT pointed at the mount. First, check that the ComfyUI server answers.

Code, shellcurl -sf -m 3 "${COMFYUI_URL:-http://127.0.0.1:8188}/system_stats"

AI agents are sent to AGENTS.md first, and its rules hold for people too: stop and ask when a request fits no row of the decision tree, when a precondition fails, when a render would exceed 30 minutes, or before writing more than 10 dialogue lines or 15 scenes, and never hide a failure behind a silent retry.

Sources

• AEON-7/aeon-movie-maker, the repository
• Lightricks/LTX-Video, the LTX model card
• Sister repositories: comfyui-aeon-spark, aeon-music-maker, aeon-radio-drama, aeon-music-video
• Hugging Face tokens for gated Lightricks files; Civitai account for style LoRAs