Qwen3-ASR server: transcription at 16 times real time on a DGX Spark

Every voice agent I have built in the lab has the same three organs: something that hears, a language model that thinks, and something that speaks. The thinking and the speaking were already containers on the Spark. Hearing was the gap. I wanted speech to text next to the LLM, fast enough for a phone call, speaking the HTTP dialect every client already knows, the OpenAI Whisper transcription API, so nothing downstream would need an adapter.

Qwen3-ASR server is that organ: an OpenAI-compatible /v1/audio/transcriptions server backed by Qwen3-ASR-0.6B, the smaller of Qwen's two automatic speech recognition (ASR) models. vLLM, the open-source inference engine, serves it natively on the NVIDIA DGX Spark and other Blackwell consumer GPUs. It recognizes 30 spoken languages and 22 Chinese dialects, and on the Spark it transcribes at about 16 times real time: a two-second clip comes back as text in about 120 milliseconds once the server is warm.

Why the image is lean, and what is in it

The image, ghcr.io/aeon-7/qwen3-asr-server:latest, is about 33 GB. Its base is the AEON vLLM core image, kin to the build in the AEON vLLM Ultimate article, which already registers the Qwen3-ASR model class. The Dockerfile adds three things: flash-attention 2, an attention kernel that works in tiles held in the GPU's fast on-chip memory, faster and lighter than the textbook form, built natively for sm_120, the Blackwell consumer compute architecture; soundfile as the primary decoder for WAV and FLAC; and PyAV, installed with --no-deps, as the fallback for MP4, M4A, WebM, MP3 and OGG.

That --no-deps is a scar. The obvious way to give vLLM audio support is pip install vllm[audio], and the runbook forbids it in any derived image, because the install re-resolves the dependency tree and downgrades two packages the base pins for speed, flashinfer-python from 0.6.9 to 0.6.8.post1 and apache-tvm-ffi from 0.1.10 to 0.1.9. Nothing fails; latency simply gets worse. The flash-attention build fails just as quietly: it takes 10 to 15 minutes for one architecture, and if it breaks, or the host is not aarch64 with sm_120, attention silently falls back to SDPA, PyTorch's slower general scaled dot-product attention, until you rebuild with a matching FLASH_ATTN_CUDA_ARCHS.

The numbers, with their context

A real-time factor, RTF, is how many seconds of audio the server gets through per second of wall-clock time, so 16x means a minute of speech in just under four seconds. The repository's performance table reads 16.04x: a 2 s mono 24 kHz WAV in, text out, 120 ms of wall time, on the Spark, one request at a time, on the hot path. With the default four slots, four 2 s clips together finish in about 140 ms, an aggregate of about 56x, and eight take about 250 ms, about 64x, at saturation. The hot path excludes the first request, the slow one: a cold start takes 30 to 90 seconds on the Spark while vLLM compiles its CUDA graphs, recorded GPU work replayed later without launch overhead.

One measured turn through the Matrix bridge shows that transcription is the cheap step of a voice loop.

The 0.6B, the 1.7B and the aligner

Qwen3-ASR-0.6B is the validated default and the only variant I have benchmarked on this image. It sits in about 5 to 10 GB of the Spark's 128 GB of unified memory, the pool the CPU and GPU share, which vLLM reserves as a fraction through --gpu-memory-utilization: 0.06 to 0.08, with the image defaulting to 0.08. Qwen3-ASR-1.7B covers the same languages, and the repository calls it the most accurate open ASR model available; it is selectable but unbenchmarked here, wants 0.14 to 0.18, about 18 to 22 GB resident, with 0.16 in the runbook, and suits offline batch work where accuracy outranks slots. Qwen3-ForcedAligner-0.6B adds word timestamps, which this image does not produce, as a separate container for up to five minutes of audio in 11 languages.

The 30 languages span English and the major European languages, Japanese, Korean, Vietnamese, Thai, Indonesian and Malay, and Arabic, Hebrew, Hindi, Bengali, Tamil and Urdu; the 22 Chinese dialects include Cantonese, Wu, Min and Hakka. Switching is a remove and a redeploy:

Code, shelldocker rm -f qwen3-asr
QWEN_ASR_MODEL=Qwen/Qwen3-ASR-1.7B GPU_MEM=0.16 bash deploy/deploy-asr.sh

Bringing it up

You need Docker with the NVIDIA runtime, at least 12 GB of memory free, and outbound HTTPS to huggingface.co for the weights and ghcr.io for the image. The compose file joins the container to the aeon-stack bridge that the LLM and TTS containers share and gives it 4 GB of shared memory:

Code, shelldocker network create aeon-stack
git clone https://github.com/AEON-7/qwen3-asr-server
cd qwen3-asr-server
docker compose up -d

/health answers 200 as soon as the process listens, before the weights are loaded, so it is no readiness signal. I poll /v1/models, which lists the model only once it is served, then transcribe:

Code, shelluntil curl -sf -m 2 http://localhost:8001/v1/models >/dev/null 2>&1; do sleep 5; done
curl -X POST http://localhost:8001/v1/audio/transcriptions -F file=@speech.wav -F model=qwen3-asr -F language=en

The answer is {"text":"The capital of France is Paris."}, and the OpenAI SDK needs only a base URL:

Code, Pythonfrom openai import OpenAI
client = OpenAI(base_url="http://localhost:8001/v1", api_key="ignored")
with open("speech.wav", "rb") as f:
    out = client.audio.transcriptions.create(model="qwen3-asr", file=f, language="en")
print(out.text)

The baked-in command is vllm serve Qwen/Qwen3-ASR-0.6B with --served-model-name qwen3-asr, the name clients send as model, --max-num-seqs 4 for the slots, raised together with memory, and --trust-remote-code, which Qwen3-ASR requires. Any WAV sample rate is resampled to 16 kHz inside, stereo is downmixed, and raw 16-bit PCM needs a WAV header first. Omit language, or send auto, and the model detects it.

What went wrong, and what I watch now

The readiness lie is half the problem. Docker's restart: unless-stopped restarts a container that crashes, not one that fails its healthcheck, so a server that is up and deaf stays that way. The Dockerfile's HEALTHCHECK probes /v1/models every 30 seconds after a 120-second grace period and fails on an empty model list, and the compose file's optional autoheal sidecar restarts a container labeled autoheal=true on that failure.

Memory is the other recurring failure. When boot dies with No available memory for the cache blocks, raise the memory fraction by 0.02 and retry; if it still fails at 0.20, lower --max-model-len from 8192 to 4096, cheap because transcription rarely exceeds 4K tokens. More than five restarts a minute is almost always memory pressure, and a plain restart never fixes it. On the Spark the fraction never goes above 0.88.

Two symptoms point at audio decoding. Empty text with HTTP 200 usually means the decode failed; the log says Failed to load audio via soundfile. A transcript taking 30 seconds or more means PyAV is decoding instead of soundfile, about ten times slower on WAVs; check import soundfile inside the container. If the log asks for vllm[audio], pull :latest again; install nothing.

Where it sits in the voice stack

Three sidecars share the aeon-stack bridge: the LLM main, a Qwen3.6-27B AEON Ultimate DFlash build whose nearest article here is Qwen 3.8 27B AEON Ultimate; this server; and Qwen3-TTS server for the voice on the way back. They also share one 128 GB Spark, and the budget is tight: the 27B LLM at 0.75 takes about 96 GB, ASR 5 to 10 GB, TTS about 4 GB and the host about 10 GB, which leaves about 10 GB of margin and is why the LLM comes up first. matrix-voip-agent answers Matrix calls and pipes audio through all three, and on its host the client side is four lines of .env:

Code, shellSTT_BACKEND=qwen
ASR_ENDPOINT=http://${SPARK_HOST}:8001/v1/audio/transcriptions
ASR_MODEL=qwen3-asr
ASR_LANGUAGE=en

The integration notes cover OpenWebUI, Home Assistant and custom clients, and the repository's agents.md is a bring-up runbook an AI agent can follow. Create agentic personas is the natural next step once the agent can hear, and everything I publish is listed in the Aeon Forge catalog.

Sources

• Repository and image: AEON-7/qwen3-asr-server, ghcr.io/aeon-7/qwen3-asr-server:latest
• Model cards and paper: Qwen3-ASR-0.6B, Qwen3-ASR-1.7B, Qwen3-ForcedAligner-0.6B, arXiv 2601.21337
• Sibling TTS server: AEON-7/qwen3-tts-server
• Voice bridge: AEON-7/matrix-voip-agent
• LLM main of the stack: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash
• Spark memory cap note: AEON-7/Qwen3.6-NVFP4-DFlash