Skip to content
AWS Machine Learning Blog· Dmitry Soldatkin·· 1 h agoAI score58

Amazon SageMaker AI launches cross-Region inference routing for endpoints and inference components

Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI

AI summary

Amazon SageMaker AI introduced cross-Region inference routing, a new inference router resource that gives applications one invocation path across endpoints and inference components in multiple AWS Regions and accounts. The router prefers destinations in its own Region, shifts new requests elsewhere when local destinations fail, retries a real-time request once only when the model did not run, and returns traffic gradually after recovery. It is generally available in most commercial AWS Regions except me-south-1 and me-central-1, with no additional charge for the routing itself; response streaming is not supported and real-time requests must complete within about a minute.

Source:AWS Machine Learning Blog · aws.amazon.com