Modal now lets each Function queue up to 1 million inputs, up from 2k, with higher .spawn() rate limits and FunctionCall results retained for 7 days. The update also adds ephemeral apps via app.run() inside containers, Git commit info for apps in the CLI and dashboard, and a new TensorRT-LLM example serving LLMs at under 400 ms latency.
Modal cut GPU, CPU and memory prices by up to 65%, and Team plan users can now run up to 50 GPUs concurrently. The Modal SDK is now on conda-forge starting with version 0.73.46, and a version 1.0 client release is planned. Modal also published an LTX-Video image-to-video example and launched a new logo.
Modal shipped a refreshed logs dashboard with time-range input filtering, live container profiling for debugging stuck containers or inputs, and Debug Shells that run commands like vim, ps, and strace inside running containers. Region selection is now available on all plans, priced with a multiplier on base prices, and the client adds OIDC authentication for cloud bucket mounts via the new oidc_auth_role_arn field.
Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.
Why it matters: Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.
Modal now offers NVIDIA L40S GPUs at $1.95/hr, with 48GB of memory, up to 40% faster performance on memory-bound tasks and over 100% speedup on compute-bound jobs versus its A10 GPUs. The platform also added Proxy Auth tokens for gating web endpoints, a Filesystem API for Sandboxes, and disk snapshotting that lets users branch new Sandbox instances from snapshots to eliminate cold starts. Modal additionally announced completion of its SOC 2 Type 2 certification.
Modal announced the general availability of Sandboxes, its primitive for safely running untrusted code from LLMs, users, or third-party sources in an isolated environment separate from other Modal resources. Sandboxes offer a simple exec API, runtime-configurable environments via the Image API, and filesystem snapshots for persistence. Modal also upstreamed support into SWE-bench, where a --modal flag lets the Verified benchmark's 500 tasks run in about 7 minutes, and cited Quora's Poe, Codegen, and Relevance AI as customers.
Modal shipped a product update round led by a major upgrade to its beta memory snapshotting feature, which it says can cut cold start times by half for models like Stable Diffusion. The platform also added OpenID Connect support for short-lived identity tokens, eStargz image compression for faster builds from registries like ECR and Docker Hub, and is expanding async job queues toward a 1 million input limit, up from 2k.
Modal now offers NVIDIA L40S GPUs to all users at $1.95/hr, positioned as a faster, larger-memory alternative to its most popular inference accelerator, the A10. The L40S doubles GPU RAM to 48GB, letting models like Flux.1-schnell run in 16bit precision on a single GPU, and delivers roughly 40% speedup on memory-bound jobs and over 100% on compute-bound 16bit Tensor Core jobs.
Modal shipped static IP proxies that route outbound traffic through workspace-exclusive IPs, plus a native Slack integration for real-time app alerts and a live usage dashboard tracking resources across all workspace apps. The update also adds GPU fallbacks for functions, expanding the pool of resources a function can be scheduled on, and client changes including `modal shell` container IDs and memory snapshotting for parametrized functions.
Modal has added native dynamic batching support, letting inference functions group requests via a single @modal.batched() decorator with max_batch_size and wait_ms parameters. On OpenAI's Whisper large v3 running on an A10G, a max_batch_size of 64 raised throughput from ~1.2 to ~3.3 requests per second per container and cut inference cost by 65%.
Modal shipped app rollbacks that let users revert a deployed app to a previous version from the dashboard or CLI, plus a @batched decorator that accumulates requests into dynamically sized batches to trade latency for throughput and GPU utilization. Sandbox objects can now forward TCP ports to the internet and exec additional processes, and logs can be exported to an OpenTelemetry provider. CPU cores and higher-end GPUs also got cheaper.
Cursor has implemented a shadow workspace that lets its AI apply edits in a hidden Electron window and read back lint results without touching the user's editing session. The post outlines the design goals of LSP availability, independence, privacy, concurrency and speed, and explains why copying a TextModel or running separate language servers failed. It also sketches a future kernel-level folder proxy for runnability, noting that macOS and Windows lack a viable FUSE path and that Rust LSP support is currently unavailable.
Modal is cutting prices on CPUs and some of its most powerful GPUs by 15-30%, citing increased competition in the GPU supply pool and the launch of more powerful GPUs over the past year. The company is also changing CPU/memory billing to charge max(reservation, usage) per running container, with a minimum default reservation of 0.125 CPU cores for all Functions. The Team plan rises to $250/mo with unlimited seats and $100/mo in credits, with new pricing taking effect in September for affected Starter and Team/Enterprise users.
Modal's monthly product update adds a Datadog integration that forwards application and audit logs with a single click, plus cloud region selection for functions to meet regulatory, egress cost, or database-locality needs. The company also reworked its function call system for higher throughput with significantly lower overhead, targeting low-latency real-time production workloads, and overhauled the app details page with time-interval zoom, container status, and image file system views.
Modal's monthly product update adds writable mounts for AWS S3 and Cloudflare R2 buckets, a @web_server decorator for hosting non-ASGI/WSGI web servers on a specified port, and Okta single sign-on for enterprise plan users. A volume viewer in the web dashboard lets users explore volume file systems, and the latest client adds multi-file app composition via .include, partitioned queues, and stdin support for Sandboxes.
Modal now supports WebSocket connections, letting functions on its platform maintain persistent, bidirectional links between client and server instead of per-request HTTP connections. Each WebSocket connection counts as a single input, so functions not bound by CPU/GPU should allow concurrent inputs or Modal spins up a new container per connection. Modal auto-scales WebSocket handlers, targeting real-time streaming, progress updates on long-running tasks, and hosting frameworks like ComfyUI, Streamlit and Gradio.
Modal has added WebSocket support to its functions, enabling out-of-the-box hosting of frameworks like Streamlit, plus interactive commands via `modal container exec` for inspecting and debugging active containers. H100 GPUs are now available on Modal, and the client update adds `@build` lifecycle hooks, automatic background commits for Volumes, and expanded CLI commands including get, put and rm.
Modal has made NVIDIA H100 GPUs available to all users, priced at $7.65 per hour per GPU. NVIDIA benchmarks show the H100 delivers up to 4x training speedups and up to 30x inference speedups versus the A100, with 80 GB of on-chip DRAM and up to 8 GPUs linked via NVLink at 3.6 TB/s. Users enable it by setting "H100" as the GPU type in the Modal decorator.