AWS makes SageMaker HyperPod model caching generally available

Amazon Web Services said model caching for SageMaker Inference on HyperPod is now generally available in all regions where HyperPod is offered. The feature pre-loads model weights and inference-server container images onto cluster nodes before pods are scheduled, so pods read from local NVMe storage at roughly 7 GB/s instead of pulling weights and images over the network. AWS said benchmarks across 57-145 GB models showed about 60 percent faster scale-out with weights caching, while image caching removed over two minutes of cold image-pull time. AWS said the first cache population still requires a remote download, and weights caches are per-node, so NVMe consumption scales with node count.