Live
Amazon SageMaker Inference supports multi-instance GPU
Trending storyPast story
Amazon SageMaker Inference supports multi-instance GPU
1 reports1 outlets12 days agoUpdate
The story so far
No overview yet — start with the coverage timeline below.
Coverage timeline
Follow the coverage to see every side of the story.
7 Nov
- AWS Machine Learning BlogSageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
Amazon SageMaker Inference now supports GPU partitioning, dividing physical GPUs into independent partitions via NVIDIA Multi-Instance GPUs (MIG), where each partition receives a fixed share of GPU computing units and dedicated memory blocks for isolation. Deploy more models in isolation and choose the appropriate size of GPU units on demand to reduce costs, with a maximum of 7 partitions per GPU, no need to build Kubernetes yourself.