Skip to content
Live
Trending storyPast story

Amazon SageMaker Inference supports multi-instance GPU

1 reports1 outlets12 days agoUpdate

The story so far

No overview yet — start with the coverage timeline below.

Coverage timeline

Follow the coverage to see every side of the story.

7 Nov
  1. AWS Machine Learning Blog
    SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

    Amazon SageMaker Inference now supports GPU partitioning, dividing physical GPUs into independent partitions via NVIDIA Multi-Instance GPUs (MIG), where each partition receives a fixed share of GPU computing units and dedicated memory blocks for isolation. Deploy more models in isolation and choose the appropriate size of GPU units on demand to reduce costs, with a maximum of 7 partitions per GPU, no need to build Kubernetes yourself.