Tag
modelopt
Also taggeddgx-sparknvfp4awqcalibrationdflasheaglegemma-4gemma4
Posts and projects tagged modelopt
- modelopt-fast-moe cuts AWQ calibration time for MoE models at NVFP4
modelopt-fast-moe replaces the one-sample-at-a-time calibration loop that leaves a GPU mostly idle while nvidia-modelopt quantizes a Mixture-of-Experts model to NVFP4. What broke on a 26B MoE, what the batched loop fixes, and what it still leaves to you.
The Aeon Forge6 min read - Gemma 4 31B DECKARD HERETIC at NVFP4 with DFlash on a DGX Spark
A dense, uncensored Gemma 4 31B with thinking, quantized to NVFP4 and served through a DFlash drafter on one DGX Spark. Single-stream decode went from 11 to 39 tokens per second, 32 concurrent requests reach 427, and reasoning and tool calling stay intact.
The Aeon Forge11 min read