MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
MACaQ is a post-training quantization method for MXFP4 W4A4, which reduces costs while maintaining accuracy in language and diffusion models by using online activation transformations and Hessian-aware refinement to independently quantize each linear layer. MACaQ achieves only a 2.3% end-to-end overhead increase on Qwen3-32B with vLLM, has a lower perplexity than DuQuant++ and MR-GPTQ, and reaches the best perplexity of Llama-3-8B in 11 times shorter quantization time.
Published on Nov 7
Authors:
,
,
,
Abstract
MACaQ is an MXFP4 W4A4 post-training quantization method that quantizes each linear layer independently using online activation transforms and Hessian-aware refinement, reducing cost while preserving accuracy across language and diffusion models.
Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. MACaQ applies a per-channel scale and a block-Hadamard transform online to each layer's activations, while absorbing the corresponding inverse transforms into its weights. Retaining these transforms online avoids cross-layer dependencies and enables a simple streaming implementation of the quantization process, in which only one layer must reside in memory at a time. To improve over round-to-nearest quantization, we introduce Flip-Refinement, a Hessian-aware coordinate-descent procedure that greedily moves individual FP4 codes to adjacent representable values. Despite executing the activation transforms at inference time, MACaQ adds only 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM. MACaQ reaches lower perplexity than DuQuant++ and MR-GPTQ, and already matches their best perplexity on Llama-3-8B in 11 times less quantization time. We further apply MACaQ without architecture-specific modifications to an image diffusion model, illustrating the generality of its layer-local design and show quantization of a 122B parameter model in 64 minutes, using only 32.5GB GPU RAM.
View arXiv page View PDF Project page GitHub 3 Add to collection
Get this paper in your agent:
hf papers read 2611.10649
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.10649 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.10649 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.10649 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
Source: Hugging Face Daily Papers · huggingface.co