AWS adds GPU sharing for SageMaker real-time inference

AWS says Amazon SageMaker AI now lets customers run several models on one GPU by dividing it into as many as seven isolated partitions. Previously, customers serving small and mid-size models had to allocate an entire accelerator to each model, leaving some paid capacity unused. The feature uses NVIDIA Multi-Instance GPU technology to give each partition its own memory and compute. Customers configure partitions when creating an endpoint with inference components and continue invoking models by name without changing application code. Availability covers P4d, P5, G7 and G7e instances in seven regions across the United States, Asia Pacific and Europe.