AWS demonstrates SageMaker HyperPod Data Capture flywheel
Start here
AWS engineers published a walkthrough of a model-improvement loop run entirely on one SageMaker HyperPod cluster, using the new Data Capture capability, which records live inference inputs and outputs to Amazon S3 at three independent tiers. In the example, a Qwen2.5-3B-Instruct support-ticket router misroutes tickets for a newly launched air-conditioner line; Athena queries surface the gap, a SQL policy relabels the captured tickets, and a 60-second LoRA fine-tune on a second GPU raises held-out accuracy from 83.9% to 100% without regressing existing categories. The authors report no measurable latency difference between requests sent directly to vLLM and those routed through the capture proxy. Data Capture is available today on HyperPod clusters that run the inference operator; users enable it by adding a dataCapture section to their InferenceEndpointConfig or JumpStartModel resource. Closing the loop on the same cluster requires adding the amazon-sagemaker-hyperpod-training-operator EKS add-on. The training job installs three Python packages at start, and the training node needs outbound internet access, for example through a NAT gateway, as does pulling the vllm/vllm-openai image from Docker Hub.
Generated from reports · updated 41 min ago
Report timeline
Follow the reports to see the event from different sides.
- AWS Machine Learning BlogAWS walks through a model improvement flywheel with SageMaker HyperPod Data Capture
AWS engineers demonstrate a complete loop on a single SageMaker HyperPod cluster using the new Data Capture capability, which records live inference inputs and outputs to Amazon S3 at three independent tiers. In the walkthrough, a Qwen2.5-3B-Instruct support-ticket router misses a newly launched air-conditioner line, Athena queries surface the blind spot, a SQL policy relabels the captured tickets, and a 60-second LoRA fine-tune on a second GPU lifts held-out accuracy from 83.9% to 100% with no regression on existing categories. The authors report no measurable latency difference between requests sent directly to vLLM and through the capture proxy.
Heat for this event
Not enough continuous observations yet to draw a trend.