使用两个NVIDIA DGX Station通过vLLM运行GLM-5.3
When One GPU Isn’t Enough: Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM
两台NVIDIA DGX Station通过ConnectX-8 RDMA连接，使用vLLM和张量并行处理，成功运行了175B参数的GLM-5.3 NVFP4模型。
Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations...
Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations connected over ConnectX-8 RDMA serve one model through a single endpoint, with tensor parallelism splitting the model weights and computation across both GPUs. The NVIDIA GLM-5.3 NVFP4 model, a quantized version of Z.ai’s mixture-of…
来源: NVIDIA Technical Blog · developer.nvidia.com