The day's essential AI stories, from the labs, startups and researchers shaping it.
vLLM BlogFeatured6060The vLLM-Omni team has integrated MiniMax H3-Image into NVIDIA Vera Rubin. With a four-card SP4 configuration, the native 4096×4096 image-to-image short prompt benchmark median latency is 0.87 seconds, and the throughput of four copies SP1×DP4 reaches 6115 images per hour.
Hacker News · 300+Featured7878llama-modchip releases a set of ggml/llama.cpp core patches and quantization transcoding tools, compressing the 284.33B DeepSeek-V4-Flash to 56.56 GiB (1.71 bpw), which can run on a pure CPU machine with 64 GB memory, with a peak memory of 57.61 GiB under 256K context and no need for swap.
Why it matters: The project provides a quantization scheme and core patches for running 284B MoE on 64 GB memory, along with measured data for each hardware, which can be used to determine the feasible boundary for pure CPU deployment.
vLLM BlogFeatured6060The vLLM team used P/D separation and Wide-EP to serve DeepSeek-V4.1-Flash on a single NVIDIA GB300 NVL72 rack, achieving 954K tokens/s per GPU and 196 tokens/s for single-user P90 on SemiAnalysis AgentX, which is 7.9 times the early single-node results.
Why it matters: The vLLM team has released per-item optimization and measured data for serving DeepSeek-V4.1-Flash on GB300 NVL72, which can be compared with DeepSeek's own engine DeepInfer.
vLLM BlogFeatured6262vLLM team releases the second part of GLM-5.3 optimization, deploying on 16 H200s with 1 pre-filled node and 1 decoding node, using PCP to split pre-fill by token and DCP to split KV cache by token, allowing 1M context to fit into the pre-fill node
Why it matters: The vLLM team has dissected how PCP, DCP, and Hybrid HiSparse can be combined on 16 H200s, allowing readers to understand the memory and pre-fetch bottlenecks of long-context deployment.
