AWS Launches SageMaker JumpStart Model Leaderboard
Start here
AWS launched the Amazon SageMaker JumpStart Model Leaderboard inside SageMaker Studio, letting users compare JumpStart models side by side on quality, speed and cost. The leaderboard reports AWS-run scores on BIG-Bench Hard, GPQA Diamond and MMLU-Pro, along with measured time to first token, throughput and instance cost. At launch it covers 70+ models from providers including Meta, Google, Qwen, IBM and Arcee, with more to be added over time. Not every JumpStart model has benchmark scores yet, and the launch benchmarks focus on LLMs. Estimated cost reflects SageMaker instance cost only, based on the on-demand hourly rate for the configuration's default instance type. Using the leaderboard requires access to SageMaker Studio and permission to deploy JumpStart models.
Generated from reports · updated 20 h ago
Report timeline
Follow the reports to see the event from different sides.
- AWS Machine Learning BlogAmazon SageMaker JumpStart Model Leaderboard launches in SageMaker Studio
AWS launched the Amazon SageMaker JumpStart Model Leaderboard in SageMaker Studio, letting users compare JumpStart models side by side on quality, speed, and cost. It reports AWS-run scores on BIG-Bench Hard, GPQA Diamond, and MMLU-Pro, plus measured time to first token, throughput, and instance cost. At launch it covers 70+ models from providers including Meta, Google, Qwen, IBM, and Arcee.
Heat for this event
- Current, comparable range
- 6
- Peak, comparable range
- 10November 19 08:00
- Change in 24 hours
- –
The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.