Skip to content
Hot eventWatching

Google releases Streaming DiLoCo in MaxText v0.2.5

1 reports1 reporting sourcesUpdated 1 d ago

Start here

Report summary

Google implemented Streaming DiLoCo in MaxText v0.2.5, its open-source JAX training framework, splitting model parameters into P shards and synchronizing only one shard per step, thereby completing distributed LLM pretraining over low-bandwidth wide-area networks.

From Google Developers Blog

Report timeline

Follow the reports to see the event from different sides.

November 19
  1. Google Developers Blog
    Google Implements Streaming DiLoCo in MaxText to Enable Cross-Data-Center Distributed LLM Training

    Google implemented Streaming DiLoCo in MaxText v0.2.5, its open-source JAX training framework, splitting model parameters into P shards and synchronizing only one shard per step, thereby completing distributed LLM pretraining over low-bandwidth wide-area networks.

Heat for this event

Not enough continuous observations yet to draw a trend.