Skip to content
Hot eventDeveloping

AWS brings cross-Region inference routing to SageMaker AI

2 reports1 reporting sourcesUpdated 53 min ago

Start here

AI digest

AWS released cross-Region inference routing in Amazon SageMaker AI as a new inference router resource that gives applications a single invocation path across endpoints and inference components in multiple AWS Regions and accounts. The router prefers destinations in its own Region, moves new requests elsewhere when local destinations fail, retries a real-time request once only if the model did not run, and returns traffic gradually after recovery. AWS said the feature is generally available in most commercial Regions except me-south-1 and me-central-1, for real-time and asynchronous inference, with no additional charge for the routing itself; customers still pay standard SageMaker AI pricing for the endpoints and inference components that serve requests, and standard CloudWatch rates for router metrics. Response streaming is not supported, and real-time requests must complete within about a minute. Customers can cap remote traffic in the receiving endpoint's CrossRegionTrafficConfig as a per-instance request rate and the largest share of endpoint capacity remote traffic may use; local traffic never counts against the cap. Salesforce's Agentforce Foundation team said it adopted the routing to serve in-house and open source models across 30+ Regions and hundreds of AWS accounts, placing it in its Internal Gateway after the data residency check so residency-tagged requests stay pinned to their in-Region endpoint.

Generated from reports · updated 44 min ago

Report timeline

Follow the reports to see the event from different sides.

November 20
  1. AWS Machine Learning Blog
    Amazon SageMaker AI launches cross-Region inference routing for endpoints and inference components

    Amazon SageMaker AI introduced cross-Region inference routing, a new inference router resource that gives applications one invocation path across endpoints and inference components in multiple AWS Regions and accounts. The router prefers destinations in its own Region, shifts new requests elsewhere when local destinations fail, retries a real-time request once only when the model did not run, and returns traffic gradually after recovery. It is generally available in most commercial AWS Regions except me-south-1 and me-central-1, with no additional charge for the routing itself; response streaming is not supported and real-time requests must complete within about a minute.

November 19
  1. AWS Machine Learning Blog
    Salesforce scales global model serving with cross-Region inference routing in Amazon SageMaker AI

    Salesforce's Agentforce Foundation team adopted cross-Region inference routing in Amazon SageMaker AI to serve in-house and open source models across 30+ global Regions and hundreds of AWS accounts. The router sends each request to an endpoint that can serve it, shifting traffic away from destinations hit by 5xx errors, throttles, and timeouts, typically starting within about a minute. Salesforce integrated it into its Internal Gateway after the data residency check, so residency-tagged requests stay pinned to their in-Region endpoint.

Heat for this event

Current, comparable range
10
Peak, comparable range
10November 20 03:00
Change in 24 hours
–

The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.