Modal adds memory snapshots to cut serverless cold starts by 2.5x
Memory snapshots: Checkpoint/restore for sub-second startup
Nội dung này chưa có bằng ngôn ngữ của bạn; đang hiển thị bằng English.
Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.
Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.
Nguồn:Modal Blog · modal.com