Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4
Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4
NVIDIA CUDA 13.4 released, enabling GPU sharing in Kubernetes through memory-enforced mechanisms, allowing a single GPU to be split into hardware-isolated instances for multiple AI workloads, improving resource utilization.
Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads...
Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads with the infrastructure they already have. NVIDIA GPUs already support several approaches to sharing. Multi-instance GPU (MIG) splits a GPU into hardware-isolated instances that are transparent to the application…
Source: NVIDIA Technical Blog · developer.nvidia.com