Tag
gemma4
Also taggedawqcalibrationdgx-sparkmixture-of-expertsmodeloptnvfp4quantization
Posts and projects tagged gemma4
- modelopt-fast-moe cuts AWQ calibration time for MoE models at NVFP4
modelopt-fast-moe replaces the one-sample-at-a-time calibration loop that leaves a GPU mostly idle while nvidia-modelopt quantizes a Mixture-of-Experts model to NVFP4. What broke on a 26B MoE, what the batched loop fixes, and what it still leaves to you.
The Aeon Forge6 min read