Tag
mixture-of-experts
Also taggeddgx-sparknvfp4dflashspeculative-decodinguncensoredvllmabliterationagentic-coding
Posts and projects tagged mixture-of-experts
- Serving Qwen 3.6 35B-A3B heretic at NVFP4 with DFlash on a DGX Spark
How I serve an uncensored Qwen 3.6 35B-A3B at NVFP4 with DFlash speculative decoding on one DGX Spark: the seven container fixes, the mixture-of-experts angle, the drafter, the measured numbers and the exact command that works today.
The Aeon Forge11 min read - Ornith 1.0 35B AEON Ultimate keeps the coding and drops the refusals
I abliterated DeepReinforce's Ornith 1.0 35B, an agentic coding MoE, and kept the coder: 0 of 80 refusals, pass@1 identical to the base, a three-way BF16, NVFP4 and DFlash benchmark on a DGX Spark, and the quick start to serve it.
The Aeon Forge10 min read - modelopt-fast-moe cuts AWQ calibration time for MoE models at NVFP4
modelopt-fast-moe replaces the one-sample-at-a-time calibration loop that leaves a GPU mostly idle while nvidia-modelopt quantizes a Mixture-of-Experts model to NVFP4. What broke on a 26B MoE, what the batched loop fixes, and what it still leaves to you.
The Aeon Forge6 min read - Gemma 4 26B-A4B at NVFP4: a mixture-of-experts recipe for a DGX Spark
How I serve Gemma 4 26B-A4B, a mixture-of-experts model at NVFP4, on one DGX Spark with a DFlash drafter: the production recipe, the bench recipe behind 816 tok/s at c64, and the drafter defects that made my earlier numbers wrong.
The Aeon Forge10 min read