# Amazon SageMaker Inference adds GPU partitioning with Multi-Instance GPU

> Original title: SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

- Source: AWS Machine Learning Blog
- Published: 2026-11-06T18:01:00.000Z
- Bad Influence: https://badinfluence.news/items/y1bokqlz85mcr3k8328xmhnz7
- Original: https://aws.amazon.com/blogs/machine-learning/sagemaker-inference-now-supports-multi-instance-gpu-to-maximize-gpu-utilization-for-model-endpoints/

## Summary

Amazon SageMaker Inference now supports GPU partitioning, using NVIDIA Multi-Instance GPU to divide one physical GPU into isolated partitions that each host a model. The feature is available on P4d, P5, G7, and G7e instance families in seven AWS Regions, and customers are billed for the full GPU instance. In the blog's benchmark, one mig-3g.40gb partition of an H100 delivered 53 to 89 percent more output tokens per second than a whole A100-40GB at high concurrency, and was cheaper per million output tokens above a traffic-dependent crossover point.

## Why it matters

The post gives per-GPU-type partition profiles and a benchmark against a whole A100, showing when a fractional GPU beats a full one.