Skip to content
Live
Trending storyPast story

Ant Group and vLLM community open-source Humming core library

1 reports1 outlets29 days agoUpdate

The story so far

AI overview

Ant Group and the vLLM community have open-sourced the high-performance kernel library Humming for general quantized inference, supporting NVIDIA GPUs and devices like the T-Head Zhenwu 810E, offering multiple quantization schemes that outperform existing implementations on multiple representative workloads.

AI-written from the reports · Updated 27 days ago

Coverage timeline

Follow the coverage to see every side of the story.

21 Oct
  1. vLLM Blog
    Announcing Humming: Fast, Flexible Kernels for Quantized Inference

    Ant Group and the vLLM community open-source Humming, a high-performance quantized inference kernel library. Humming supports NVIDIA GPUs starting from the Turing architecture and CUDA-compatible devices like the T-Head Zhenwu 810E, providing extensive quantization support covering various integer and floating-point types, bit widths, and scaling schemes. It significantly outperforms existing implementations such as vLLM Marlin, FlashInfer, and DeepGEMM on multiple representative workloads.