Tag
vllm
Also taggeddgx-sparkdflashnvfp4speculative-decodingabliterationblackwelldockergb10
Posts and projects tagged vllm
- Qwen3-ASR server: transcription at 16 times real time on a DGX Spark
Qwen3-ASR on vLLM gives a DGX Spark a self-hosted, Whisper-compatible transcription endpoint for 30 languages and 22 Chinese dialects at about 16 times real time. Here is the pre-built image, the 0.6B and 1.7B variants, and the lessons from running it.
The Aeon Forge7 min read - Qwen 3.8 27B AEON Ultimate: the current flagship and how to serve it
Qwen3.8-27B AEON Ultimate Uncensored is the model I now serve on one Spark, two Sparks and the RTX box: what coherent abliteration removes and keeps, the NVFP4-MIXED weight lattice, the three seat recipes with their commands, and the rules that keep them running.
The Aeon Forge14 min read - Serving Qwen 3.6 35B-A3B heretic at NVFP4 with DFlash on a DGX Spark
How I serve an uncensored Qwen 3.6 35B-A3B at NVFP4 with DFlash speculative decoding on one DGX Spark: the seven container fixes, the mixture-of-experts angle, the drafter, the measured numbers and the exact command that works today.
The Aeon Forge11 min read - Nemotron 3 Nano Omni AEON Ultimate, served uncensored on a DGX Spark
An abliterated Nemotron 3 Nano Omni that reasons over text, images and audio on one DGX Spark. How the twelve-dimension refusal removal was found, what NVFP4 costs against BF16, what the source-built vLLM image patches, and how to deploy it.
The Aeon Forge9 min read - Gemma 4 31B DECKARD HERETIC at NVFP4 with DFlash on a DGX Spark
A dense, uncensored Gemma 4 31B with thinking, quantized to NVFP4 and served through a DFlash drafter on one DGX Spark. Single-stream decode went from 11 to 39 tokens per second, 32 concurrent requests reach 427, and reasoning and tool calling stay intact.
The Aeon Forge11 min read - Gemma 4 26B-A4B at NVFP4: a mixture-of-experts recipe for a DGX Spark
How I serve Gemma 4 26B-A4B, a mixture-of-experts model at NVFP4, on one DGX Spark with a DFlash drafter: the production recipe, the bench recipe behind 816 tok/s at c64, and the drafter defects that made my earlier numbers wrong.
The Aeon Forge10 min read - DeepSeek-V4 on two DGX Sparks: an experimental TP=2 image
An experimental vLLM image that serves DeepSeek-V4-Flash across two DGX Sparks at tensor parallel 2 over RoCE: the seven GB10 gaps it patches, why PIECEWISE graphs are the rule, what the numbers do and do not prove, and a two-node quick start.
The Aeon Forge7 min read - DDTree on vLLM: candid research notes on tree speculation for Qwen3.6
DDTree puts tree-shaped speculative decoding into vLLM for Qwen3.6 on a DGX Spark. These lab notes show what works, what breaks inside a hybrid attention model, the numbers so far, and why production should stay on DFlash.
The Aeon Forge8 min read - AEON vLLM Ultimate RTX: vLLM for the RTX 5090 and Blackwell sm_120
A slim vLLM image for the RTX 5090 and RTX PRO 6000 Blackwell, why it must never be crossed with the DGX Spark image, what changes on a 32 GB card, four serving recipes for a 27B NVFP4 model, and the WSL2 fault that passed every test.
The Aeon Forge9 min read - AEON vLLM Ultimate for ROCm: early access on vLLM 0.31
AEON vLLM Ultimate for ROCm is a vLLM 0.31 image that serves Qwen3.8-27B AEON ULTIMATE in MXFP4 and MXFP6 on AMD GPUs. It covers what is built, why the stock path decoded at 1.7 tok/s, what is still unmeasured, and how to test the 2x R9700 recipe.
The Aeon Forge6 min read - AEON vLLM Ultimate: one serving image for the DGX Spark
AEON vLLM Ultimate is vLLM built from source for the DGX Spark's GB10 and shipped as one 18.3 GB image that serves the whole AEON fleet with speculative decoding and NVFP4 weights. Here is why it exists, what broke on the way, and how to pull and run it.
The Aeon Forge14 min read - AEON Bench recipe lab: tuning vLLM flags one at a time on a DGX Spark
The AEON Bench recipe lab treats a serve recipe as an experiment: change one flag, bench, read the curve. The knobs in order of effect, the speculative decoding trap behind 33 versus 517 tok/s on a DGX Spark, the 816 tok/s recipe to steal, and four traps that fake a result.
The Aeon Forge11 min read