Amazon ECS adds shared GPU capacity for concurrent tasks

AWS says Amazon Elastic Container Service now lets multiple tasks run concurrently on one NVIDIA GPU through NVIDIA Multi-Process Service. The capability is available in all AWS commercial and AWS GovCloud (US) Regions on ECS Managed Instances and ECS on Amazon EC2. Customers specify each container's GPU memory requirement and can cap its share of computing capacity. ECS places tasks on GPUs with available capacity without requiring application code changes. AWS says sharing can improve utilization and reduce infrastructure costs for workloads that use only part of a GPU, including small-model inference and embedding generation. On Managed Instances, ECS also monitors the service that enables sharing and replaces unhealthy instances, moving tasks to healthy capacity.