AWS Machine Learning Blog· Apoorva Chandra·· 23 h agoAI score38
Salesforce scales global model serving with cross-Region inference routing in Amazon SageMaker AI
How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI
AI summary
Salesforce's Agentforce Foundation team adopted cross-Region inference routing in Amazon SageMaker AI to serve in-house and open source models across 30+ global Regions and hundreds of AWS accounts. The router sends each request to an endpoint that can serve it, shifting traffic away from destinations hit by 5xx errors, throttles, and timeouts, typically starting within about a minute. Salesforce integrated it into its Internal Gateway after the data residency check, so residency-tagged requests stay pinned to their in-Region endpoint.
Source:AWS Machine Learning Blog · aws.amazon.com