v0.34.0
v0.34.0
GitHub Copilot v0.34.0 released, providing code generation, review, security, and collaboration features, supporting custom integrations and advanced security features.
v0.34.0
1,061 commits from 412 contributors (120 new). Thank you!
TL;DR
- 10 new models (Mistral Large 4, MiniMax-M3.1-Flash, Nemotron-3.5-Transcribe, and more!)
- Extensible KV cache is on by default: vLLM sizes the KV cache from memory actually free after warmup, so most models get a bigger cache and more throughput.
- Model Runner V1 is gone. Model Runner V2 is now the only GPU runner.
- Big speedups on Blackwell and MI355X: up to 1.95× decode (Step5 on GB200), 1.6× throughput (GLM-5.3-Flash), 1.24–1.40× faster long-prompt TTFT (GLM-5.3).
- Upgrade check: default block size is now 64, ROCm 7.2 images are gone, and
huggingface_hub >= 2.0.0is required. See Before you upgrade.
Before you upgrade
Important
These changes can break existing setups.
| Change | What to do | PR |
|---|---|---|
| Model Runner V1 removed | Remove VLLM_USE_V2_MODEL_RUNNER=0; it now fails at startup. Port V1 logits processors to vllm.v1.worker.gpu.sample.logits_processor. |
#63373 |
Default --block-size is 64 (was 16) |
Pass --block-size 16 to keep the old behavior. ROCm paged attention still uses 16. |
#63231 |
Extensible KV cache on by default; gpu_memory_utilization defaults to 1.0 |
To keep the old sizing, pass --gpu-memory-utilization 0.9 or --no-enable-extensible-kv-cache. |
#56492 |
| ROCm 7.2 artifacts dropped | Use the ROCm 10.0 wheels and images. -rocm72 is no longer published. |
#61939 |
huggingface_hub >= 2.0.0 required |
Also requires tokenizers >= 0.23.3 and timm >= 1.0.30. |
#64779 |
QuTLASS / fp_quant removed |
fp_quant checkpoints no longer load. |
#65230 |
| Marlin W4A8 removed | Use --quantization humming with VLLM_HUMMING_INPUT_QUANT_CONFIG instead of VLLM_MARLIN_INPUT_DTYPE. |
#65542 |
| Keye-VL and Keye-VL-1.5 removed | — | #64287 |
All breaking changes, deprecations and new defaults are listed in the full changelog.
New models
| Model | Notes | PR |
|---|---|---|
| Mistral Large 4 | 1T total / 52B active, FP8 and NVFP4. Runs on NVIDIA Blackwell/Hopper and AMD MI350. | #64534 |
| MiniMax-M3.1-Flash | Blackwell NVFP4 and ROCm kernels, DSpark draft | #64111 |
| NVIDIA Nemotron-3.5-Transcribe | 30B-A4B speech model: audio chat and /v1/audio/transcriptions |
#64282 |
| Ricursive Ripple-65B Preview | BF16 and NVFP4, ripple reasoning and tool parsers |
#65745 |
| Zyphra ZAYA1 | ZAYA1-8B, ZAYA1-74B-preview | #65302 |
| Baidu Unlimited-OCR-2 | DASH sparse attention | #64016 |
| Liquid AI LFM3, LFM3-VL, LFM3-Audio | #65826, #65827, #65828 | |
| NVIDIA Canary-1B-v2 | ASR and X↔En translation | #64215 |
| Ming3-ASR | #63585 | |
| NVIDIA Parakeet TDT | Through the vllm-parakeet-plugin | #65005 |
Highlights
Extensible KV cache, on by default
- On CUDA and ROCm, the KV cache is sized from memory actually free after CUDA graph capture and warmup, not from an estimate (#56492).
- Result: a larger cache (e.g. MiniMax-M3 on H200: 433k → 672k tokens), so throughput is higher and TTFT lower at high concurrency.
- Trade-off: per-token decode latency can rise on some models (TPOT +6–18% on some H200 models at concurrency 128). Opt out with
--gpu-memory-utilization 0.9or--no-enable-extensible-kv-cache. - Hardened with fixes for allocator fragmentation, startup OOMs and KV connectors (#67296, #67485, #66731, #66523, #66197, #65137, #66605, #66674).
Model Runner V2 only
- Model Runner V1 is removed (#63373).
- Dual-batch overlap now runs on MRV2, including with speculative decoding and DCP (#61464).
Speculative decoding
- Suffix decoding runs on MRV2 with a GPU speculator; Arctic Inference is no longer needed (#60339).
- Custom speculators: declarative configs plus a
register_speculative_methodplugin API (#63205). - CUDA graphs for MRV2 draft-model speculators (#60989).
- Adaptive verification works with XPress (#65175).
- Draft KV cache matches the target's dtype, so an FP8 target gets an FP8 draft cache (#65198).
Performance
| Improvement | Hardware | Gain | PR |
|---|---|---|---|
| Step5 NVFP4 MoE on TRTLLM-Gen | 4×GB200 | up to 1.95× decode | #65152 |
| GLM-5.3-Flash DeepGEMM FP8 MegaMoE | NVIDIA | 1.6× output throughput vs DEP4 | #60895 |
| Muse Spark Open specialized decode (opt-in) | GB300 | 1.5× single-stream decode | #64077 |
| Token-sharded sparse-MLA prefill under TP (default) | GB200 | GLM-5.3 TTFT 1.24–1.40× faster at 32K–128K | #63956 |
| KV-balanced MLA decode for Kimi-K3 (opt-in) | GB300 | 1.34–1.59× decode attention | #63638 |
| Qwen3.5 on breakable CUDA graphs (no torch.compile) | NVIDIA | 3.0–8.8% lower decode ITL | #51012 |
| FlyDSL decode monokernel, Muse Spark Open DSpark | 8×MI355X | +35–57% interactivity at conc. 1–8 | #63440 |
AMD ROCm
- Release images and wheels are ROCm 10.0 only (#61939).
- MoRI EPv2 via
--all2all-backend mori(#61361). - Kimi-K3 specialized decode on MI355X, opt-in (#61435).
- TRITON_ATTN is the default attention backend on RDNA3 and RDNA4 (#60418).
Serving and operations
- Checkpoint/restore (CRIU +
cuda-checkpoint) through/checkpoint/prepare,/checkpoint/statusand/checkpoint/restore, in both the Python and Rust frontends (#62570). - Fault-tolerant DP+EP scale-down: dead ranks are removed and their experts redistributed without a restart (#46370).
- Best-effort NUMA binding on by default (#61258).
- Opt-in cache-aware admission ordering (#54625).
- Rust frontend: stateless Responses API (#63902) and
n > 1(#65625).
Install
Python wheels
| Platform | Install |
|---|---|
| CUDA 13.0 (PyPI) | pip install vllm |
| CUDA 13.0 (uv) | uv pip install vllm --torch-backend=auto |
| ROCm 10.0 | pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.34.0/rocm100 |
| XPU | uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.34.0/xpu --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match |
Docker images
| Platform | Image | Short alias |
|---|---|---|
| CUDA 13.0 (default) | vllm/vllm-openai:v0.34.0 |
vllm/vllm:v0.34.0 |
| CUDA 13.0 + DeepEP over AWS EFA | vllm/vllm-openai:v0.34.0-efa |
vllm/vllm:v0.34.0-efa |
| ROCm 10.0 | vllm/vllm-openai-rocm:v0.34.0 |
vllm/vllm:v0.34.0-rocm |
| CPU | vllm/vllm-openai-cpu:v0.34.0 |
vllm/vllm:v0.34.0-cpu |
| XPU | vllm/vllm-openai-xpu:v0.34.0 |
vllm/vllm:v0.34.0-xpu |
latest tags work the same way (e.g. vllm/vllm:latest-rocm). The default CUDA image now includes the AWS EFA userspace (libfabric, aws-ofi-nccl). The -efa image adds DeepEP built for EFA and routes NCCL over EFA by default (#65576).
The Assets section below has the source tarball and the CUDA 13.0 (x86_64, arm64), CPU (x86_64, arm64, macOS) and XPU (x86_64) wheels.
Full changelog
Model support
- Mistral Large 4 (#64534) runs on MRV2 with:
- sparse MLA plus the DSA indexer, and cross-layer KV sharing;
- DSpark, DCP/PCP, NIXL P/D and HiSparse.
- Other new models:
- DeepSeek-V4 / V4.1
- Prefill gathers compressed KV once per KV-source group: 38 → 4 gathers per step (#59131).
- Decoder replay graphs share the model graph pool: prefill graph memory 15.52 → 9.88 GiB per rank on GB200 (#63155).
fused_inv_rope_fp8_quant3.5× faster, prefill 6% faster (#63540).- DSpark/MTP memory savings of about 512 MiB per GPU (#63996).
- Engram host tables shared across DP ranks (#62359).
- DeepGEMM sparse MQA logits on the FP8 indexer cache (#62011).
- Packed multi-image vision encoding, 1.6–2.6× (#65030).
- LoRA on the language model, compressor and
wo_alayers (#57245, #64880). - DSpark with PCP (#61994); PCP with PP or DP (#63098).
- Fixes: sparse prefill in FULL CUDA graphs (#59986), prefill memory budgeting (#63047), MTP embedding under PP (#65816), DeepSeek-V3.2 PP (#59961).
- GLM
- GLM-5.3-Flash: DeepGEMM FP8 MegaMoE (#60895), adaptive verification (#60229), row-strided QKV into FlashKDA (#61614).
- GLM-5.3 FP8 MoE sorted-A TMA on H200, 6.0–7.6% E2E throughput (#64760).
- FP8 KV cache for DSpark and DFlash drafts (#63841, #64612, #65553).
- FP32 indexer MQA logits restored for GLM-5/GLM-5.3/HY-V4, with new
--attention-config.indexer_logits_dtype(#64580). - Fused KDA convs fix stale weights after RL refit (#60991).
- Muse Spark
- Opt-in Muse Spark Open specialized decode on GB300 (#64077).
- One RoPE cache shared across layers: 11.5 GiB per GPU saved, 14.5% more KV cache on 8×B200 (#65206).
- GEMM and communication+norm fusions (#65438, #64638); hybrid DeepGEMM MegaMoE / TRT-LLM for NVFP4 (#64525).
- CUDA routing kernel (#64553), MegaMoE JIT warmup (#63920), DSpark draft loading fix (#65125).
- MiniMax-M3 / M3.1
- Sequence parallelism and GEMM-RS for TP + MegaMoE, 14–20% lower P50 TTFT (#64659), also with PP (#65537).
- Fused norm + MXFP8 quant, 1.39–1.60× (#64852).
- Triton query-tiled sparse prefill on Hopper (#57420); SM90 split-K indexer (#64037).
- No host sync in prefill metadata (#65883); NVFP4 MegaMoE warmup (#66227); fmha_sm100 indexer updates (#66228).
- Fixes: M3.1 prefill workspace (#64856), sequence parallelism (#65496), encoder CUDA graphs (#65226).
- Kimi
- Opt-in
--attention-backend KIMI_K3_KV_BALANCED_MLAfor agentic workloads on Blackwell (#63638). - Kimi-K3 MegaMoE symmetric buffer 10.73 → 2.50 GiB per GPU at TP8 EP (#63541).
- FlashInfer fused FP8 chunked-context K/V pack (#61527).
- Fixes: KDA prefill OOM (#63989), NaNs via FlashKDA/FlashMLA pins (#64759), sleep mode (#64583, #66273).
- Kimi K3.1 groundwork landed (#66030, #66028), but K3.1 is not servable in this release.
- Opt-in
- Qwen
- Qwen3.5: breakable CUDA graphs instead of torch.compile (#51012); fused norm/gate + quant (#64509) and AR + norm + quant (#64723); GDN
in_projoverlap (#64425, #64999). - GDN prefill state gather/scatter fused for Qwen3-Next/Qwen3.5 and Kimi KDA, 1.16–1.50× (#66064).
- Qwen3.8-Flash-Next: PLE disk offload via
--engram-config '{"disk_offload": true}'(27 GiB less on DGX Spark, #59958), PP (#60234), shared QSA top-k buffer (#65619), loading fixes (#62318, #65812, #64820, #60490). - Fixes: long-context int32 overflow (#63761), multi-layer MTP (#62111).
- Qwen3.5: breakable CUDA graphs instead of torch.compile (#51012); fused norm/gate + quant (#64509) and AR + norm + quant (#64723); GDN
- Other models
- Step5 MegaMoE with PP (#65504) and KV block sizing (#65153).
- Gemma 4: multimodal encoder DP (#60314), PP (#65989), FA4 FP8 MMA on SM90 (#60372).
- Moved to the Transformers modeling backend: OLMoE (#63384), Gemma v1 and StableLM (#66393), Aria (#59767; temporarily about 11% less KV cache and 25% lower image throughput).
- Matryoshka dimensions for EmbeddingGemma 2 (#64885, #60978); RADIO C-RADIO v2/v3/v4 (#58468).
- Qwen3-ASR on the upstream Transformers config; no
--hf-overridesneeded for the ForceAligner (#66097, #63274). - Fixes: GPT-OSS MXFP4 LoRA (#65825), PLaMo3 GPTQ (#62836), Ling3 (#65088), DomynEdge (#66819).
- Multimodal
- Encoder CUDA graphs for Pixtral/Mistral3 (#56277), shared capture buffers (#65084), one replay per batch up to
encoder_cudagraph_max_token_budget(#65227). - On-device input normalization for Nemotron VL (#57924), Gemma 3 (#64487), Gemma 4 (#65577).
--mm-encoder-tp-mode datafor MiMo-V2.6 (#64179).- Fixes: FlashInfer ViT attention on GLM-4.1V/GLM-OCR/GLM-5.3-Flash (#65778), Qwen3-Omni and Qwen2.5-Omni video budgets (#66017, #66190), dots_ocr with TP (#64543), embedding-only startup (#65689), media cache eviction (#65363).
- Encoder CUDA graphs for Pixtral/Mistral3 (#56277), shared capture buffers (#65084), one replay per batch up to
Engine core
- Extensible KV cache
- On by default on CUDA and ROCm (#56492).
- Headroom for measured allocator fragmentation, validated on Qwen3.5-397B-A17B-FP8 at TP8+EP on 8×H200 (#67296).
- Fixes: sizing and startup OOMs (#67485, #66523, #66731, #66197, #65554), connectors that failed to start (#65225), up to 84% undersized caches on DGX Spark (#65413),
--max-model-len auto(#65463). - Startup warning when an explicit
gpu_memory_utilizationwithholds memory from it (#66704); docs on when to set it (#67071).
- Model Runner V2
- The only runner after MRV1's removal (#63373); DBO with spec decode and DCP (#61464).
- Separate prefill (token) and decode (request) CUDA graph capture sizes (#63406).
- Staged writes copied straight into UVA (#63633); idle sampler writes skipped, about 10–11% faster small-batch decode (#62748).
- Sampler warmed up after graph capture to stop memory growth under load (#62560).
- Lower peak memory for chunked prompt logprobs (#64841, #64914); Triton ReplaySSM (#65033).
- NVIDIA Confidential Computing decode fixes (#59981).
- Speculative decoding
- Suffix decoding (#60339) and custom speculators (#63205) on MRV2; draft-model CUDA graphs (#60989).
- XPress with adaptive verification (#65175); adaptive verification draft budgets on a Triton kernel (#63421).
- Batch-sharded draft logits (#63554); DSpark sequential blocks (#63886).
- DSpark sliding-window MLA draft layers (#63795) under DCP (#63796); fused Markov sampling, 42–50% less sampling time (#64476);
draft_lm_head_quantization(#64368). - Draft KV cache dtype aligned with the target (#65198).
- Trace replay with ngram/EAGLE3/MTP, 1.47–2.03× (#63275); 3× faster resample kernel (#65100).
- Fixes: DFlash acceptance of 0 with FULL CUDA graphs (#64059), a resample race (#66048), acceptance-estimator cancellation (#65229), standalone draft prefill under FULL graphs (#65804), grammar-invalid drafts (#61557).
- Scheduler and engine overhead
- Prompt token ids kept as int32 buffers: 40× less per-request CPU on a 128K prompt (#62702).
- Per-request id lists kept out of the cyclic GC (#65300); 45–50% less KV block allocation CPU (#65831).
- New requests and aborts handled while waiting for model output (#65466), with a cheaper wait loop (#65773).
- Opt-in cache-aware admission via
--cache-aware-admission-window(#54625). --max-num-inflight-kv-loadsstarts async KV loads while the batch is full (#65319), with cached reservations: 47.2 → 5.4 ms per schedule (#65285).- A multimodal item is no longer split by a prefix hit (#65271).
- Startup and compile
- No openai/fastapi imports in EngineCore and workers (#65423, #65589).
- CUDA graph memory profile skipped when its estimate is disabled (#64622).
- Device-neutral compile cache via
VLLM_COMPILE_ON_ONE_RANK, on by default (#63247); all graph-affecting config fields hashed into compile cache keys (#65652). --shuffle-safetensors-filesfor large EP startups (#42837).BasevLLMParameter's Python__torch_function__dropped: GLM-5.2-FP8 TTFT −6.1% on GB300 (#63785).
- Lifecycle
- Process checkpoint/restore over HTTP (#62570); opt-in multi-worker CUDA checkpoint sleep (#64186).
- Best-effort NUMA binding by default;
--no-numa-bindto disable (#61258). - Sleep mode with exact-size pinned backups, up to about 578 GiB less host memory per node on Mistral Large 4 TP8 (#64630); level 2 discards only parameters (#66010, #66165, #59994).
- Fast Start weight-cache restarts for online FP8/MXFP8, ModelOpt CUTLASS FP8/NVFP4/block-FP8 and AWQ/GPTQ Marlin (#63672, #63754, #64119, #64182), with pluggable shared warmup-cache stores (#65168).
WatermarkConfig.enforce(#62966).
- RL and weight loading
VLLM_RELOAD_MODE=directwrites checkpoint tensors straight into the live model (#64336, #64352, #65281).- Every load and update finishes through
finalize_model_weights(#65914, #64357, #64692, #57678). - Load plans for weight-transfer engines (#65235); per-update NCCL source ranks for PP trainers (#62557).
- Sparse weight patches validated in one pass, 40–48% shorter pause (#66021).
- Offloaded KV tier survives pause/sleep (#57810); partial stacked weights rejected (#63752).
- LoRA
- MoE-LoRA shrink/expand kernels up to 64 tokens (#64653); embedding LoRA scaled by
embed_scale(#62470). - Rejected: adapters on unwrapped modules (#63489), non-linear expert placement with EP (#62797).
- Fixes: reloads with requests in flight (#64616), KV-sharing fast prefill (#61162), SEQ_CLS heads (#63286), fused MoE expand rounding (#63894), Mamba-1 (#64681).
- MoE-LoRA shrink/expand kernels up to 64 tokens (#64653); embedding LoRA scaled by
- Batch invariance: LFM2 ShortConv (#62367), FlashInfer FP8 bmm and W4A16 NVFP4 (#62192), GPTQ (#60479), FA sliding window (#65627),
aten::mmout_dtype overloads (#60047).
Large-scale serving
- Expert parallelism and fault tolerance
- DP+EP scale-down via
POST /v1/fault_tolerance/apply(#46370); MoRI EPv2 (#61361). - Opt-in
symm_memEPLB weight-migration backend (#62009). --enable-ep-weight-filterwith EPLB: DeepSeek-R1 NVFP4 DEP8 reads 105 GiB per rank instead of 385 GiB (#64187).num_redundant_expertsdefaults to the minimum valid value (#30614).- Skipped combine staging copy in
flashinfer_nvlink_one_sided(#65498); DeepEP V2 padded rows (#54109).
- DP+EP scale-down via
- Context parallelism and sparse attention
- Token-sharded sparse-MLA prefill under TP (#63956); DCP on the FlashMLA sparse bf16 KV path (#61609).
- MLA decode up-projected before the A2A merge with
--dcp-comm-backend a2a(#64994); faster DCP sparse-indexer top-k (#65178, #64129). - Cross-layer KV cache sharing for MLA layers (#64095).
- Fixes: PP rank derivation under PCP (#65539), FlashInfer MLA DCP decode (#59392).
- KV connectors
- NIXL: push mode via
transfer_mode(#64230);ucx_backend_params(rocm_ipc on MI355X: 3.4 → 483.7 Gbps, #62905); 38% fewer hybrid SSM transfer bytes (#62317); block pairing across different P/D block sizes (#59746), with mismatched push handshakes rejected (#59747); same-host NVLink with the extensible KV cache (#65137). - Mooncake: selective leases (#63420), automatic
cache_prefix(#65160), per-GPU RDMA devices (#65076), heterogeneous-TP hybrid sharing (#54307), sleep mode (#59934), lease renewal (#64193). - Lookahead block hashes in NIXL, Mooncake and offloading connectors (#64096); UMBP standalone mode (#61795, #61793).
- Out-of-tree and LMCache MP connectors with the extensible KV cache (#66605, #66674).
- NIXL: push mode via
- KV offload
- SimpleCPUOffload with one host buffer per rank and opt-in HugeTLB pages (#60130); no zero-filled pool (#64345).
- KVCR v0.2.0 (#64224, #63889); async lookups published per request group (#57474); secondary tiers rejected on multi-node engines (#62930).
- Hardened P2P tier (#63781, #64250, #64256, #64251, #64501, #63618, #64300, #63988) and correctness fixes (#64229, #63566, #65279, #65631, #64400, #63937, #65640).
- HiSparse
- Uses the extensible KV cache by default: 16% more GPU+host KV capacity (#65214).
- Fused slot invalidation, up to 1.8% lower TPOT on GLM-5.3-NVFP4 (#60103); fewer hash probes (#61263); speculative-group prefetch (#63801).
- Platform backend for hardware plugins (#62592); scheduler deadlock fix (#62979).
- EPD and distributed
Hardware
- NVIDIA
- Step5 NVFP4 TRTLLM-Gen MoE (#65152).
- DeepSeek-V4.1: latency_specialized decode extended, 5.9–17% faster decode step on GB300 (#63921); swap-AB CuTe DSL decode attention for TP2/TP4, 1.4–2.15× (#62633); opt-in fused DSpark Markov sampling, 1.6–3.9× (#64442).
- Sparse-indexer top-k reads BF16 logits directly (#62172, #64138); single-launch fused prefill top-k (#60618).
- GLM-5.3 TMA MoE projections on H200 and SM100 (#63820, #65746); SM90 native small-head sparse FlashMLA, +10% on GLM-5.3-Flash TP4 (#64510).
- CuTe DSL BF16x3 router GEMM for fp32 routers on Hopper (#61903); fused GDN MTP decode for DSpark K=8, +18% on B300 (#65874).
- GLM-5.3-Flash on 2× DGX Spark, 15% lower TPOT (#62956); faster weight loading on integrated GPUs (#58868).
- Rubin locality-domain NVFP4 MoE (#61685); Jetson Thor (SM110) fixes (#62133, #65050, #62275).
- AMD ROCm
- ROCm 10.0 only (#61939); AITER v0.1.28.post1 (#63774, #65937); MoRI EPv2 (#61361).
- MonoKernel decode for Muse Spark Open DSpark verify (#63440), DeepSeek-V4.1-Flash at TP8 (#64101) and the MiniMax-M3.1 DSpark drafter, +10% P90 interactivity (#65386).
- Kimi-K3 specialized decode (#61435), AITER-tuned prefill GEMMs: mean TTFT 5.35 → 4.05 s (#63730), opt-in FlyDSL FP8 MLA prefill (#59124).
- DeepSeek-V4-Pro P/D over MoRIIO (#48989); Step-5-Preview (#61837) and Reflection Beam 1.0 (#62057) on ROCm.
- RDNA: TRITON_ATTN by default (#60418), AITER Triton kernels on RDNA3 (#52970), HIP GDN prefill backend for gfx1151 (#56307), opt-in W4A16 dense prefill cache: TTFT −22 to −26% on Strix Halo (#60710).
- Fixes: Kimi-K2.6 int4 prefill regression on Triton 3.8 (#62565), MI355 extensible KV cache OOMs (#65418).
- Intel XPU
- CPU
- Other: out-of-tree layer overrides via
vllm.model_executor.hw_agnostic.resolve(#60218); shared routed-expert row restorers (#64689); Ascend Triton fix (#64013).
Quantization
- New
--quantization lut_b_moe: about 3.14-bit MoE experts with MXFP8 activations, native on Rubin (#62759). w4afp8checkpoints on the W4A8 FP8 MoE path (#64410).- Block Hadamard input rotation with online
mxfp4_hadamard32and Quark online-rotation checkpoints (#62750). - CSF-compressed NVFP4 block scales in ModelOpt (#64312); activation quant overrides for online MXFP4 (#51808).
- DeepSeek-V4.1 MXFP8 linears on cuBLASLt (#64815); 3× faster MXFP4 activation quant (#63101); faster FP8 quant on SM100+ (#61641).
- QuIP NVFP4 on Humming (#65230).
- Fixes: NVFP4 NaN logits (#63382, #61755), Humming FP8 (#60163, #56606), hadacore (#63514).
API and frontend
- New endpoints and options
/checkpoint/prepare,/checkpoint/status,/checkpoint/restore(#62570).thinking_token_budgeton/v1/responses(#63896);/v1/completionsinvllm run-batch(#64097).- Streaming derender on
/inference/v1/generate(#63103). - Anthropic Messages API:
min_tokens(#65459) andthinking.type=between_tools(#65954). vllm:request_render_duration_secondshistogram (#60213).
- Tool calling and reasoning
- New
granite_toolcall_parserfor Granite 4.2 (#65216); llama3_json moved to the ParserEngine (#63569). - Incremental streaming for Kimi K3 (#62203),
tool_choice="required"(#60882), DeepSeek V4/V3.2 (#58780), GLM-4.7 (#64535), Qwen3 (#64675). - Fixes: DeepSeek V4.1 (#62517, #64971, #62729, #62647), GLM-4.7 (#64615), Step3 (#65274), FunctionGemma (#63992), union/enum string parameters (#64518).
- New
- Behavior
- Corrupt or oversized images and disabled local media return 400 instead of 500 (#63433, #60308, #64032).
- Responses accepts incomplete assistant/reasoning history, e.g. from Codex (#35508).
- Multi-template processors use the default template (#63606); an empty
toolsarray is accepted like OpenAI (#65547). - The first streamed usage reports cached prompt tokens (#65036); plain completion stream frames render about 9× faster (#52875).
- Rust frontend
- Stateless Responses API (#63902) with namespace tools (#65853);
n > 1(#65625);--skip-tokenizer-init(#64697). - Hybrid and external DP load balancing (#64451, #64450); native GLM-5.3 chat rendering (#62424).
- Prompt block hashing off the engine's GIL (#63842);
--reasoning-effort-map(#62312). --enable-log-outputs,--max-log-len,--root-path,--log-error-stack(#63769, #64223, #64222, #65001).- More request, response and metric parity with Python (#65293, #65294, #65623, #65624, #64218, #64526, #63964, #65636).
- Stateless Responses API (#63902) with namespace tools (#65853);
Security, dependencies and build
- Security
- Dependencies
- PyTorch stays at 2.14.0. FlashInfer 0.7.3 (#65930).
huggingface_hub >= 2.0.0,tokenizers >= 0.23.3,timm >= 1.0.30(#64779);transformers >= 5.16.1, < 5.23.0(#64049, #64529);mistral-common >= 1.13.0(#64169).- DeepEP 2.1 (#59844); DeepGEMM pin updates (#64411, #64580, #65494); FlashMLA and FlashKDA pins (#61971, #64759).
- NIXL 1.5.2 (#64579, #64585); mooncake-transfer-engine 0.3.14 (#65576); Run:ai Model Streamer 0.17.1 (#63174);
fastokens >= 0.3.5forVLLM_USE_FASTOKENS(#65679). - ROCm: AITER v0.1.27 → v0.1.28.post1 (#63774, #65937). XPU: PyTorch 2.15 (#63977), triton-xpu 3.9.0 (#65047), vllm_xpu_kernels 0.1.17 (#60609). CPU: zentorch 2.14.0.2 (#65669).
- Build
- x86_64 CUDA wheel about 82 MB (about 25%) smaller, kernels unchanged (#65518).
- Default CUDA image installs the AWS EFA userspace; the
-efaimage now only adds DeepEP for EFA (#65576); nightly EFA images asvllm/vllm-openai:efa-nightly(#64028). - One ROCm build and release pipeline, ROCm 10.0 only (#61939).
- Source builds fetch
triton_kernelsfrom a tarball (#65162) and ship tuned kernel configs in the wheel (#64063). - CUDA
x.yfarches no longer fall back to a lower minor (#66016).
All breaking changes, deprecations and new defaults
- Model Runner V1 removed (#63373).
VLLM_USE_V2_MODEL_RUNNERandVllmConfig.use_v2_model_runnerare gone; setting the env var to anything but1fails at startup.- Dropped with it: the V1 logits-processor API (
vllm.v1.worker.gpu.sample.logits_processoris the only interface), thecustom_classspeculative method,use_heterogeneous_vocab,disable_padded_drafter_batch,parallel_draftingoutside DFlash/DSpark, PP withexternal_launcher,NGramPerReqLogitsProcessor, and x86 CPUs without Triton. - Deprecated:
parallel_drafting,capture_torch_profiler,enable_layerwise_nvtx_tracing.
- Default
--block-sizeis 64 instead of 16 (#63231). Backends with a preferred block size still override it (ROCm paged attention keeps 16), and an explicit--block-sizestill wins. gpu_memory_utilizationdefaults to 1.0 when unset with the extensible KV cache on (#56492). An explicit value in (0.8, 1.0) that withholds 1 GiB or more logs a warning (#66704).- ROCm 7.2 wheels and
-rocm72images are no longer published (#61939). - Removed:
- QuTLASS and the
fp_quantmethod (#65230). - Marlin INT8/FP8-activation (W4A8) paths and
VLLM_MARLIN_INPUT_DTYPE; use--quantization hummingwithVLLM_HUMMING_INPUT_QUANT_CONFIG(#65542). - Keye-VL and Keye-VL-1.5 (#64287);
disable_mtp_completeness_check(#49459); therefresh_derived_weightshook (#65914).
- QuTLASS and the
- Dependencies:
huggingface_hub >= 2.0.0(#64779);mistral-common >= 1.13.0(#64169); XPU requires vllm_xpu_kernels 0.1.17 (#60609). - Model and plugin code: all
process_weights_after_loadinghooks share one signature and the loader runs every module's hook (#57678); configcompute_hash()replaced bycompile_factors()/hash_config()(#65652). - API behavior:
- llama3_json no longer streams tool arguments incrementally (#63569).
- Canary transcription without
languagereturns 400 (#65445). - An empty
toolsarray is accepted in both frontends (#65547, #65854). /v1/systemonerejects request-supplied chat templates without--trust-request-chat-template(#64503).- LoRA adapters on modules vLLM does not wrap are refused (#63489); misconfigured NCCL weight transfer raises (#57022).
vllm-rs serveno longer reads--data-parallel-rankfromVLLM_DP_RANK(#65369).- AITER fused shared experts with MiniMax M3 and DeepSeek V4 need
VLLM_ROCM_USE_AITER=1(#61457).
- Still deprecated:
--mamba-cache-modehas no effect; its removal was postponed (#64838). - New defaults:
- Extensible KV cache (#56492); block size 64 (#63231); best-effort NUMA binding (#61258).
- Token-sharded sparse-MLA prefill at 1024 or more prefill tokens (#63956); MLA decode up-projection before the A2A merge (#64994).
VLLM_COMPILE_ON_ONE_RANK(#63247); minimum valid EPLBnum_redundant_experts(#30614).- HiSparse on the extensible KV cache (#65214); encoder CUDA graphs up to
encoder_cudagraph_max_token_budget=5120, eager above (#65227). - TRITON_ATTN on RDNA3/RDNA4 (#60418); MRV2 on non-Triton CPUs (#61995);
float32Mamba SSM cache for Nemotron Omni (#63218); Zipf routing invllm bench forward(#64167).
Contributors
120 new contributors: welcome!
- @0xDexe made their first contribution in https://github.com/vllm-project/vllm/pull/65274
- @akote123 made their first contribution in https://github.com/vllm-project/vllm/pull/61593
- @akshay-babbar made their first contribution in https://github.com/vllm-project/vllm/pull/57990
- @aleksandr-inherent made their first contribution in https://github.com/vllm-project/vllm/pull/64583
- @amenomori made their first contribution in https://github.com/vllm-project/vllm/pull/63973
- @Anchen798 made their first contribution in https://github.com/vllm-project/vllm/pull/60252
- @Anieca made their first contribution in https://github.com/vllm-project/vllm/pull/65271
- @AnvitDevadiga made their first contribution in https://github.com/vllm-project/vllm/pull/59198
- @asm582 made their first contribution in https://github.com/vllm-project/vllm/pull/66165
- @bandham-manikanta made their first contribution in https://github.com/vllm-project/vllm/pull/63614
- @bantianyue made their first contribution in https://github.com/vllm-project/vllm/pull/64170
- @bavinainuganti9 made their first contribution in https://github.com/vllm-project/vllm/pull/60314
- @bogdan-petkovic made their first contribution in https://github.com/vllm-project/vllm/pull/64734
- @chao-xu-spec made their first contribution in https://github.com/vllm-project/vllm/pull/65087
- @ChengYM5685 made their first contribution in https://github.com/vllm-project/vllm/pull/61361
- @chennyiiis made their first contribution in https://github.com/vllm-project/vllm/pull/63076
- @chf003 made their first contribution in https://github.com/vllm-project/vllm/pull/63348
- @CJJ1008 made their first contribution in https://github.com/vllm-project/vllm/pull/62478
- @coderfeli made their first contribution in https://github.com/vllm-project/vllm/pull/64008
- @CodersAcademy006 made their first contribution in https://github.com/vllm-project/vllm/pull/38741
- @DemisEom made their first contribution in https://github.com/vllm-project/vllm/pull/65423
- @DongjiGao made their first contribution in https://github.com/vllm-project/vllm/pull/64282
- @drago1234 made their first contribution in https://github.com/vllm-project/vllm/pull/64013
- @dudilester made their first contribution in https://github.com/vllm-project/vllm/pull/62881
- @ekintel made their first contribution in https://github.com/vllm-project/vllm/pull/64829
- @emmericp made their first contribution in https://github.com/vllm-project/vllm/pull/64622
- @engint-cerebras made their first contribution in https://github.com/vllm-project/vllm/pull/60487
- @erezzarum made their first contribution in https://github.com/vllm-project/vllm/pull/64579
- @ericschreiber made their first contribution in https://github.com/vllm-project/vllm/pull/64279
- @eugr made their first contribution in https://github.com/vllm-project/vllm/pull/64782
- @G-haoyu made their first contribution in https://github.com/vllm-project/vllm/pull/57662
- @GCSimba made their first contribution in https://github.com/vllm-project/vllm/pull/64935
- @giannis2two made their first contribution in https://github.com/vllm-project/vllm/pull/62557
- @Gin-Sin made their first contribution in https://github.com/vllm-project/vllm/pull/64898
- @glenmessenger made their first contribution in https://github.com/vllm-project/vllm/pull/60899
- @guanzheh made their first contribution in https://github.com/vllm-project/vllm/pull/63540
- @GuSangmo made their first contribution in https://github.com/vllm-project/vllm/pull/63738
- @hechengkai made their first contribution in https://github.com/vllm-project/vllm/pull/64188
- @hellozmz made their first contribution in https://github.com/vllm-project/vllm/pull/64037
- @hemuku90 made their first contribution in https://github.com/vllm-project/vllm/pull/64032
- @heyemlee made their first contribution in https://github.com/vllm-project/vllm/pull/64616
- @Hiro208 made their first contribution in https://github.com/vllm-project/vllm/pull/56508
- @hojuna made their first contribution in https://github.com/vllm-project/vllm/pull/64986
- @Hritik003 made their first contribution in https://github.com/vllm-project/vllm/pull/64020
- @Hrushikesh-ramilla made their first contribution in https://github.com/vllm-project/vllm/pull/66012
- @hyunnnchoi made their first contribution in https://github.com/vllm-project/vllm/pull/59999
- @ilyas-mallah made their first contribution in https://github.com/vllm-project/vllm/pull/60438
- @isztldav made their first contribution in https://github.com/vllm-project/vllm/pull/61609
- @Jackmeson1 made their first contribution in https://github.com/vllm-project/vllm/pull/64757
- @jackyYang6 made their first contribution in https://github.com/vllm-project/vllm/pull/65817
- @Jaeyeong-CHOI made their first contribution in https://github.com/vllm-project/vllm/pull/60177
- @jbelofamd made their first contribution in https://github.com/vllm-project/vllm/pull/62905
- @jbottleson-nvidia made their first contribution in https://github.com/vllm-project/vllm/pull/63218
- @jh-nv made their first contribution in https://github.com/vllm-project/vllm/pull/63817
- @JimmyToluene made their first contribution in https://github.com/vllm-project/vllm/pull/65692
- @jiuyueQ made their first contribution in https://github.com/vllm-project/vllm/pull/64300
- @JohnZhao01 made their first contribution in https://github.com/vllm-project/vllm/pull/62729
- @jonahsamost made their first contribution in https://github.com/vllm-project/vllm/pull/60229
- @KaloyanNaumov made their first contribution in https://github.com/vllm-project/vllm/pull/63566
- @karverma-amd made their first contribution in https://github.com/vllm-project/vllm/pull/64094
- @kazutoiris made their first contribution in https://github.com/vllm-project/vllm/pull/65174
- @kbenkhaled made their first contribution in https://github.com/vllm-project/vllm/pull/56102
- @lance5890 made their first contribution in https://github.com/vllm-project/vllm/pull/65557
- @lHrHenry233 made their first contribution in https://github.com/vllm-project/vllm/pull/65812
- @Liamup777 made their first contribution in https://github.com/vllm-project/vllm/pull/65834
- @ltllt1 made their first contribution in https://github.com/vllm-project/vllm/pull/62647
- @luckyq made their first contribution in https://github.com/vllm-project/vllm/pull/62613
- @lvvgang made their first contribution in https://github.com/vllm-project/vllm/pull/42837
- @majq92 made their first contribution in https://github.com/vllm-project/vllm/pull/64543
- @Manfredss made their first contribution in https://github.com/vllm-project/vllm/pull/60163
- @MayaJShetty made their first contribution in https://github.com/vllm-project/vllm/pull/64551
- @Medhavin38 made their first contribution in https://github.com/vllm-project/vllm/pull/65001
- @mirza-halilcevic made their first contribution in https://github.com/vllm-project/vllm/pull/65428
- @MrGeva made their first contribution in https://github.com/vllm-project/vllm/pull/65883
- @my-git9 made their first contribution in https://github.com/vllm-project/vllm/pull/64471
- @nbommi19 made their first contribution in https://github.com/vllm-project/vllm/pull/63018
- @Neuro1729 made their first contribution in https://github.com/vllm-project/vllm/pull/60978
- @neverhook made their first contribution in https://github.com/vllm-project/vllm/pull/63064
- @NikitaLogvinenko made their first contribution in https://github.com/vllm-project/vllm/pull/57167
- @nkuhn-vmw made their first contribution in https://github.com/vllm-project/vllm/pull/60490
- @nrenvo made their first contribution in https://github.com/vllm-project/vllm/pull/62497
- @nsusanto made their first contribution in https://github.com/vllm-project/vllm/pull/53166
- @Nuriel32 made their first contribution in https://github.com/vllm-project/vllm/pull/62529
- @nv-nikitaj made their first contribution in https://github.com/vllm-project/vllm/pull/63382
- @obendidi made their first contribution in https://github.com/vllm-project/vllm/pull/59383
- @paolo-perrone made their first contribution in https://github.com/vllm-project/vllm/pull/64097
- @Prakash-sa made their first contribution in https://github.com/vllm-project/vllm/pull/63853
- @qqaatw made their first contribution in https://github.com/vllm-project/vllm/pull/65494
- @rexlee8776 made their first contribution in https://github.com/vllm-project/vllm/pull/62500
- @roborluo made their first contribution in https://github.com/vllm-project/vllm/pull/62632
- @Rockyeast made their first contribution in https://github.com/vllm-project/vllm/pull/63864
- @rudrakshkarpe made their first contribution in https://github.com/vllm-project/vllm/pull/63189
- @RuibinCheung made their first contribution in https://github.com/vllm-project/vllm/pull/63637
- @saichowdary007 made their first contribution in https://github.com/vllm-project/vllm/pull/53011
- @saifltr made their first contribution in https://github.com/vllm-project/vllm/pull/63992
- @sarankumar1325 made their first contribution in https://github.com/vllm-project/vllm/pull/64666
- @seanbell made their first contribution in https://github.com/vllm-project/vllm/pull/64820
- @shoutoutuoadi325 made their first contribution in https://github.com/vllm-project/vllm/pull/55669
- @Shr3yash made their first contribution in https://github.com/vllm-project/vllm/pull/59341
- @shustovTE made their first contribution in https://github.com/vllm-project/vllm/pull/66048
- @srkim-business made their first contribution in https://github.com/vllm-project/vllm/pull/62318
- @subin9 made their first contribution in https://github.com/vllm-project/vllm/pull/63286
- @suvigyav-cohere made their first contribution in https://github.com/vllm-project/vllm/pull/62966
- @tevinshi2015 made their first contribution in https://github.com/vllm-project/vllm/pull/57519
- @Than0316 made their first contribution in https://github.com/vllm-project/vllm/pull/64937
- @UsamaKenway made their first contribution in https://github.com/vllm-project/vllm/pull/62275
- @vcdlk made their first contribution in https://github.com/vllm-project/vllm/pull/65866
- @VidyasagarDudekula made their first contribution in https://github.com/vllm-project/vllm/pull/62265
- @wguo7 made their first contribution in https://github.com/vllm-project/vllm/pull/64984
- @world-executed made their first contribution in https://github.com/vllm-project/vllm/pull/63137
- @xinyuangui2 made their first contribution in https://github.com/vllm-project/vllm/pull/63548
- @xu797 made their first contribution in https://github.com/vllm-project/vllm/pull/65127
- @xutingl made their first contribution in https://github.com/vllm-project/vllm/pull/65725
- @xzwgit made their first contribution in https://github.com/vllm-project/vllm/pull/60762
- @yeh-sudo made their first contribution in https://github.com/vllm-project/vllm/pull/65050
- @yiminghub2024 made their first contribution in https://github.com/vllm-project/vllm/pull/64180
- @zhenwenqi2024 made their first contribution in https://github.com/vllm-project/vllm/pull/62592
- @ziywang50 made their first contribution in https://github.com/vllm-project/vllm/pull/65620
- @zw-g made their first contribution in https://github.com/vllm-project/vllm/pull/63606
- @zylen-det made their first contribution in https://github.com/vllm-project/vllm/pull/63292
All 412 contributors
@BugenZhao, @Thangnguyenvn98, @AndreasKaratzas, @njhill, @hmellor, @khluu, @aoshen02, @Juntian777, @mgoin, @NickLucche, @gau-nernst, @okorzh-amd, @yewentao256, @vadiklyutiy, @yma11, @zyongye, @Pb314314, @jeejeelee, @WoosukKwon, @fxmarty-amd, @waizuichougou, @stefankoncarevic, @xsxszab, @djramic, @ZJY0516, @vMaroon, @MatthewBonanni, @Fangzhou-Ai, @liranschour, @TheEpicDolphin, @chaojun-zhang, @gcanlin, @whx-sjtu, @Alex-Wengg, @roberteg16, @subhashpolisetti, @hshrivastava-droid, @hao-aaron, @mkhazraee, @divakar-amd, @Isotr0py, @seungrokj, @jperezdealgaba, @aarushjain29, @vllmellm, @bzsuni, @liusy58, @zhenwei-intel, @Etelis, @rasmith, @Ankur-singh, @reidliu41, @jzakrzew, @ZhenchengLin, @charxwu, @xinli-sw, @subin9, @namgyu-yun, @frida-andersson, @dilberx, @josh-schell-amd, @simon-veitner-redhat, @akii96, @cjackal, @chaunceyjiang, @wzhao18, @chunfangamd, @robertgshaw2-redhat, @cpersson-amd, @CJJ1008, @yzong-rh, @zhenwenqi2024, @johncalesp, @SherifWaly, @gc-fu, @Vegetog, @Manfredss, @sohom-cs, @albertoperdomo2, @Dao007forever, @Alex-ai-future, @taneem-ibrahim, @gty111, @bogdan-petkovic, @jiuyueQ, @noooop, @lucamotz, @RyanMa29, @paulpak58, @ilmarkov, @azaidy, @yifanFengg, @mispa-ms, @jh-nv, @xqoasis, @Prakash-sa, @juliendenize, @galletas1712, @mjkvaak-amd, @louie-tsai, @wtdcode, @nrenvo, @hyukjlee, @srkim-business, @hemuku90, @amenomori, @nsusanto, @zohaibm-amd, @grYe99, @LopezCastroRoberto, @janbernloehr, @RuibinCheung, @yaoyaoding, @DongjiGao, @luyixiao95, @ayush1399, @qli88, @qianlihuang, @roikoren755, @simondanielsson, @faaany, @inc-jeong, @bohnstingl, @fadara01, @spandantiwari, @BaoYunkai, @erezzarum, @mrn3088, @indivats, @LucasWilkinson, @Change72, @esmeetu, @franciscojavierarceo, @hickeyma, @zydo, @roborluo, @fuscof-ibm, @sfeng33, @gkswns0531, @anhtra3889, @lesj0610, @HollowMan6, @tomasruizt, @nien-hui, @DemisEom, @aijanai, @wxsIcey, @chao-xu-spec, @Woolgathererer, @VidyasagarDudekula, @sragrawal, @GCSimba, @ganeshr10, @Liamup777, @hj-mistral, @ivanium, @YaashuDave, @vgokhale, @shen-shanshan, @soodoshll, @Blaze-DSP, @git-jxj, @oonyshch, @zhou-yuhan, @shoutoutuoadi325, @brandonfzhang, @arpera, @zhewenl, @cleonard530, @dharak-cohere, @itej89, @Rockyeast, @he-yufeng, @xinyuangui2, @ischencheng, @NVShreyas, @GuSangmo, @neverhook, @JaredforReal, @nkuhn-vmw, @YukioZzz, @cagrikymk, @alec-flowers, @zylen-det, @xinnywinne, @wangxiyuan, @amd-xavierwang, @lucas-santos-amd, @hclsys, @stu-cao, @lijipeng787, @Sunt-ing, @Shr3yash, @giannis2two, @Hritik003, @yiminghub2024, @AlanFokCo, @lucifer1004, @bvolpato, @Yejing-Lai, @nbommi19, @paolo-perrone, @bantianyue, @AnvitDevadiga, @tarukumar, @Ronald1995, @suiyoubi, @mayuyuace, @drago1234, @mrodden, @nv-nikitaj, @jbelofamd, @hellozmz, @HwwwwwwwH, @SumanthRH, @alexm-redhat, @freyfwt, @rexlee8776, @qthequartermasterman, @coderfeli, @SIDDARTHAREDDY8, @shengliangxu, @jbottleson-nvidia, @yuanheng-zhao, @G-haoyu, @ericschreiber, @milesial, @jonahsamost, @jayzuccarelli, @sungsooha, @KaloyanNaumov, @javieratomic, @akshay-babbar, @saichowdary007, @tzielinski-habana, @CodersAcademy006, @glenmessenger, @abmfy, @orozery, @hongxiayang, @MIR-AMD, @zRzRzRzRzRzRzR, @peakcrosser7, @guan404ming, @afierka-intel, @kouroshHakha, @jikunshang, @luckyq, @chengy-sysu, @xaguilar-amd, @stecasta, @brandonpelfrey, @avininjamay8, @sarankumar1325, @hechengkai, @lizzzcai, @fangyuchu, @majiayu000, @seanbell, @karverma-amd, @wangyicong52, @ChengYM5685, @HangruiCao2000, @AMD-yanfeiwang, @TimRen01, @0z5a, @ActiveSky, @noa-neria, @Jackmeson1, @aniskumar-nv, @mindungil, @meenchen, @jthomson04, @anencore94, @SCcagg5, @heyemlee, @saifltr, @nvyutwu, @world-executed, @JulienDarve, @davidteather, @wguo7, @GodRishUniverse, @nilig, @benenzhu, @joa-stdn, @maithilijoshi20, @hyunnnchoi, @mickaelseznec, @suvigyav-cohere, @YangXu1990uiuc, @nishantrevur, @isztldav, @yeh-sudo, @chennyiiis, @ZhengGong-amd, @tevinshi2015, @Jaeyeong-CHOI, @andakai, @xu797, @zxd1997066, @Willian-Zhang, @kbenkhaled, @kliuae, @kazutoiris, @kushaldabbe, @liulanze, @UsamaKenway, @Anieca, @TQCB, @bandham-manikanta, @effi-ofer, @mirza-halilcevic, @umarkovi-amd, @nemanjaudovic, @yousafshah, @hlin99, @nliu365, @fanxingran, @afriedri, @NikitaLogvinenko, @rjrock, @0xDexe, @mosafariuk, @zixi-qi, @ltllt1, @djmmoss, @MayaJShetty, @chf003, @tlrmchlsmth, @JohnZhao01, @rudrakshkarpe, @Josephasafg, @obendidi, @Medhavin38, @Anchen798, @my-git9, @haosdent, @Than0316, @aleksandr-inherent, @tpopp, @SiluPanda, @okdshin, @fululi12, @eugr, @jamesETsmith, @askliar, @tahsintunan, @dudilester, @xzwgit, @jcotant-inferact, @Chinmay-Kulkarni-AMD, @JimmyToluene, @artem-spector, @Gin-Sin, @zexplorerhj, @jackyYang6, @qqaatw, @lHrHenry233, @zw-g, @gbyu-amd, @tryryry, @linitra24, @engint-cerebras, @katec846, @bnellnm, @MrGeva, @waynehacking8, @xutingl, @jiacao-amd, @hojuna, @MuonChaser, @bavinainuganti9, @zupengwang, @hungnnvidia, @akote123, @lance5890, @Neuro1729, @eicherseiji, @ziywang50, @lvvgang, @Hrushikesh-ramilla, @ilyas-mallah, @Mazukiri, @chenjr97, @micah-wil, @liuyumoye, @shustovTE, @emmericp, @Nuriel32, @sricursion, @Id545, @PerryZhang01, @ekintel, @GirasoleY, @guanzheh, @ShuoleiWang, @asm582, @pst2154, @mawong-amd, @majq92, @sheralskumar, @Hiro208, @vcdlk, @ZhouXiaoya25, @snadampal, @csahithi, @hnt2601, @ybgao-nvidia, @jiaqiang-dot-liu, @ketor, @ApostaC
Source: vLLM releases · github.com