Skip to content
Modal Blog· Jonathon Belotti·· 2025-01-28PicksAI score62

Modal adds memory snapshots to cut serverless cold starts by 2.5x

Memory snapshots: Checkpoint/restore for sub-second startup

AI summary

Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.

Why it matters

Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.

Source:Modal Blog · modal.com