Skip to main content

Share story

AI Infrastructure

AWS adds AZ-placement controls for SageMaker inference components

AWS adds AZ-placement controls for SageMaker inference components Image: Primary
AWS says its new SchedulingConfig controls for SageMaker AI Inference Components let customers set how model copies are distributed across instances and Availability Zones. Salesforce used the capability with balanced AZ placement and a SPREAD strategy for Agentforce deployments, AWS says, to meet its requirement that production models support two zones while retaining shared-GPU hosting. The controls can preserve configured placement during scaling and updates, though AWS says capacity constraints can still result in only partially balanced deployment unless customers reserve capacity in each zone.
Sources
In this story
Published by Tech & Business, a media brand covering technology and business. This story was sourced from AWS Machine Learning Blog and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
AI Infrastructure
AI Infrastructure

AWS launches Ray LLM containers for inference across GPU nodes

AWS has launched Ray LLM Deep Learning Containers for serving language models across multiple GPU nodes, extending its preassembled inference stack beyond single-node deployments. The containers bundle the inference engine, servin...

Infrastructure AI
Infrastructure AI

Microsoft details 96-chip Maia 300 racks and expanded inference network

Microsoft detailed a Maia 300 design that doubles accelerator density to 96 chips in a 167-kW rack and connects up to 12,288 accelerators through one network. The company says the full rack delivers up to 7 times Maia 200's token ...

Infrastructure AI
Infrastructure AI

NVIDIA contributes Vera Rubin rack designs to open infrastructure standards

NVIDIA announced contributions of Vera Rubin rack and compute tray designs to the Open Compute Project, giving equipment builders shared specifications for AI data centers. Its updated MGX 1.2 architecture supports rack deployment...

AI Infrastructure
AI Infrastructure

Microsoft plans Maia 300 rollout for production AI workloads in 2027

Microsoft announced Maia 300, its third-generation in-house AI chip, with production deployment set to begin in 2027. The company plans to scale its Maia family to multi-gigawatt deployments starting that year and make it the prim...

Infrastructure AI
Infrastructure AI

Frore introduces diamond coldplates with claimed GPU cooling gains

Frore Systems introduced LiquidJet Diamond coldplates, claiming they can handle GPU hotspots reaching 770 W/cm² and reduce GPU temperatures by 16C-22C compared with conventional coldplates. The company claims token efficiency impr...