Skip to content
Live
Trending storyPast story

vLLM released GLM-5.3 for long-context deployment optimization

1 reports1 outlets27 days agoUpdate

The story so far

AI overview

The vLLM team released the second part of their technical blog, introducing deployment optimizations for GLM-5.3 on 16 H200s: using 1 pre-filled node plus 1 decoding node, with PCP splitting pre-filled by token and DCP splitting KV cache by token, allowing 1M context to fit into the pre-filled node. The vLLM team claims that on AgentX, PCP8 × DCP8 pre-filled combined with Hybrid HiSparse decoding, at the same throughput, is 1.7–3.2 times faster per user than data parallel deployment, and also the only configuration they tested that can serve 1M token contexts. The above results require running on vLLM v0.32.0 or higher versions (vllm/vllm-openai:v0.32.0 image); if sparse_mla_force_mqa is not enabled, pre-filled batches with prompts within the 2,048 token top-k will use the dense MHA path.

AI-written from the reports · Updated 25 days ago

Coverage timeline

Follow the coverage to see every side of the story.

23 Oct
  1. vLLM BlogFeatured
    GLM-5.3 Optimizations, Part 2: PCP, DCP, and Hybrid HiSparse on 16 H200s

    vLLM team releases the second part of GLM-5.3 optimization, deploying on 16 H200s with 1 pre-filled node and 1 decoding node, using PCP to split pre-fill by token and DCP to split KV cache by token, allowing 1M context to fit into the pre-fill node