Ant Group and vLLM community open-source Humming core library
The story so far
Ant Group and the vLLM community have open-sourced the high-performance kernel library Humming for general quantized inference, supporting NVIDIA GPUs and devices like the T-Head Zhenwu 810E, offering multiple quantization schemes that outperform existing implementations on multiple representative workloads.
AI-written from the reports · Updated 27 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- vLLM BlogAnnouncing Humming: Fast, Flexible Kernels for Quantized Inference
Ant Group and the vLLM community open-source Humming, a high-performance quantized inference kernel library. Humming supports NVIDIA GPUs starting from the Turing architecture and CUDA-compatible devices like the T-Head Zhenwu 810E, providing extensive quantization support covering various integer and floating-point types, bit widths, and scaling schemes. It significantly outperforms existing implementations such as vLLM Marlin, FlashInfer, and DeepGEMM on multiple representative workloads.