Trace-Driven Study Maps AI Network Sensitivity Up to 4,096 GPUs
Start here
Tommaso Bonato and co-authors published a trace-driven study of how training and inference workloads at scales up to 4,096 GPUs respond to interconnect latency, bandwidth and jitter. The method combines signature-based trace decomposition with linear programming, preserving recorded NCCL schedules and program dependencies while sweeping network parameters and estimating a LogGP runtime lower bound. The resulting latency-bandwidth envelopes show low-sensitivity regions and performance cliffs that differ across workloads. According to the authors, this indicates when bandwidth upgrades pay off and when added network performance has little effect.
Generated from reports · updated 32 min ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AITrace-Driven Study of Network Sensitivity for AI Supercomputer Workloads up to 4,096 GPUs
A trace-driven study measures how training and inference workloads on up to 4,096 GPUs respond to interconnect latency, bandwidth and jitter. The method combines signature-based trace decomposition with linear programming to enable broad network-parameter sweeps while preserving recorded NCCL schedules and program dependencies, estimating a LogGP runtime lower bound. Latency-bandwidth envelopes reveal low-sensitivity regions and performance cliffs that vary across workloads, showing when bandwidth upgrades pay off and when extra network performance has little effect.
Heat for this event
Not enough continuous observations yet to draw a trend.