Skip to content
Only First-party#Product updates
January 28, 2025Tuesday
  1. Modal Blog62

    Modal adds memory snapshots to cut serverless cold starts by 2.5x

    Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.

    Why it matters: Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.

January 21, 2025Tuesday
  1. Modal Blog20

    Modal product updates: L40S GPUs, proxy auth tokens, and sandbox disk snapshotting

    Modal now offers NVIDIA L40S GPUs at $1.95/hr, with 48GB of memory, up to 40% faster performance on memory-bound tasks and over 100% speedup on compute-bound jobs versus its A10 GPUs. The platform also added Proxy Auth tokens for gating web endpoints, a Filesystem API for Sandboxes, and disk snapshotting that lets users branch new Sandbox instances from snapshots to eliminate cold starts. Modal additionally announced completion of its SOC 2 Type 2 certification.

  2. Modal Blog58

    Modal Sandboxes 正式可用，为 LLM 与用户代码提供隔离执行环境

    Modal announced the general availability of Sandboxes, its primitive for safely running untrusted code from LLMs, users, or third-party sources in an isolated environment separate from other Modal resources. Sandboxes offer a simple exec API, runtime-configurable environments via the Image API, and filesystem snapshots for persistence. Modal also upstreamed support into SWE-bench, where a --modal flag lets the Verified benchmark's 500 tasks run in about 7 minutes, and cited Quora's Poe, Codegen, and Relevance AI as customers.

December 28, 2024Saturday
  1. Modal Blog22

    Modal product updates: faster memory snapshotting, OIDC, async job queues

    Modal shipped a product update round led by a major upgrade to its beta memory snapshotting feature, which it says can cut cold start times by half for models like Stable Diffusion. The platform also added OpenID Connect support for short-lived identity tokens, eStargz image compression for faster builds from registries like ECR and Docker Hub, and is expanding async job queues toward a 1 million input limit, up from 2k.

December 19, 2024Thursday
  1. Modal Blog25

    Modal Adds NVIDIA L40S GPUs for AI Inference at $1.95/hr

    Modal now offers NVIDIA L40S GPUs to all users at $1.95/hr, positioned as a faster, larger-memory alternative to its most popular inference accelerator, the A10. The L40S doubles GPU RAM to 48GB, letting models like Flux.1-schnell run in 16bit precision on a single GPU, and delivers roughly 40% speedup on memory-bound jobs and over 100% on compute-bound 16bit Tensor Core jobs.

November 8, 2024Friday
  1. Modal Blog22

    Modal product updates: static IP proxies, Slack integration, live usage dashboard and GPU fallbacks

    Modal shipped static IP proxies that route outbound traffic through workspace-exclusive IPs, plus a native Slack integration for real-time app alerts and a live usage dashboard tracking resources across all workspace apps. The update also adds GPU fallbacks for functions, expanding the pool of resources a function can be scheduled on, and client changes including `modal shell` container IDs and memory snapshotting for parametrized functions.

September 16, 2024Monday
  1. Modal Blog38

    Modal adds native dynamic batching, boosting Whisper throughput 2.8x

    Modal has added native dynamic batching support, letting inference functions group requests via a single @modal.batched() decorator with max_batch_size and wait_ms parameters. On OpenAI's Whisper large v3 running on an A10G, a max_batch_size of 64 raised throughput from ~1.2 to ~3.3 requests per second per container and cut inference cost by 65%.

September 6, 2024Friday
  1. Modal Blog22

    Modal Product Updates: App Rollbacks, Dynamic Batching, Sandbox Tunnels and OpenTelemetry

    Modal shipped app rollbacks that let users revert a deployed app to a previous version from the dashboard or CLI, plus a @batched decorator that accumulates requests into dynamically sized batches to trade latency for throughput and GPU utilization. Sandbox objects can now forward TCP ports to the internet and exec additional processes, and logs can be exported to an OpenTelemetry provider. CPU cores and higher-end GPUs also got cheaper.

September 2, 2024Monday
  1. Cursor Changelog58

    Cursor details Shadow Workspace for background AI code iteration

    Cursor has implemented a shadow workspace that lets its AI apply edits in a hidden Electron window and read back lint results without touching the user's editing session. The post outlines the design goals of LSP availability, independence, privacy, concurrency and speed, and explains why copying a TextModel or running separate language servers failed. It also sketches a future kernel-level folder proxy for runnability, noting that macOS and Windows lack a viable FUSE path and that Rust LSP support is currently unavailable.

August 6, 2024Tuesday
  1. Modal Blog42

    Modal cuts CPU and GPU prices by 15-30%

    Modal is cutting prices on CPUs and some of its most powerful GPUs by 15-30%, citing increased competition in the GPU supply pool and the launch of more powerful GPUs over the past year. The company is also changing CPU/memory billing to charge max(reservation, usage) per running container, with a minimum default reservation of 0.125 CPU cores for all Functions. The Team plan rises to $250/mo with unlimited seats and $100/mo in credits, with new pricing taking effect in September for affected Starter and Team/Enterprise users.

July 9, 2024Tuesday
  1. Modal Blog22

    Modal Product Updates: Datadog Integration, Region Selection, Lower Function Latency

    Modal's monthly product update adds a Datadog integration that forwards application and audit logs with a single click, plus cloud region selection for functions to meet regulatory, egress cost, or database-locality needs. The company also reworked its function call system for higher throughput with significantly lower overhead, targeting low-latency real-time production workloads, and overhauled the app details page with time-interval zoom, container status, and image file system views.

May 7, 2024Tuesday
  1. Modal Blog18

    Modal Product Updates: Cloud Bucket Mounts, Okta SSO and More

    Modal's monthly product update adds writable mounts for AWS S3 and Cloudflare R2 buckets, a @web_server decorator for hosting non-ASGI/WSGI web servers on a specified port, and Okta single sign-on for enterprise plan users. A volume viewer in the web dashboard lets users explore volume file systems, and the latest client adds multi-file app composition via .include, partitioned queues, and stdin support for Sandboxes.

February 27, 2024Tuesday
  1. Modal Blog38

    Modal adds WebSocket support for real-time apps

    Modal now supports WebSocket connections, letting functions on its platform maintain persistent, bidirectional links between client and server instead of per-request HTTP connections. Each WebSocket connection counts as a single input, so functions not bound by CPU/GPU should allow concurrent inputs or Modal spins up a new container per connection. Modal auto-scales WebSocket handlers, targeting real-time streaming, progress updates on long-running tasks, and hosting frameworks like ComfyUI, Streamlit and Gradio.

February 15, 2024Thursday
  1. Modal Blog22

    Modal product updates: WebSocket support, interactive commands, and H100 GPUs

    Modal has added WebSocket support to its functions, enabling out-of-the-box hosting of frameworks like Streamlit, plus interactive commands via `modal container exec` for inspecting and debugging active containers. H100 GPUs are now available on Modal, and the client update adds `@build` lifecycle hooks, automatic background commits for Volumes, and expanded CLI commands including get, put and rm.

February 6, 2024Tuesday