Modal demonstrates running autoresearch agents on elastic GPUs: given Modal Skills, Claude Code ran 113 experiments over 238 GPU-hours on OpenAI's Parameter Golf challenge, finishing core training runs 5x faster than a single workstation while using a fraction of a dedicated cluster's resources. Agents provision GPUs via API calls, scaling from one to dozens of H100s and back to zero automatically.
Modal has acquired Butter, with founder Erik Dunteman and researcher Raymond Tana joining Modal's Sandbox team. The Butter team brings agent harness engineering experience, including bVisor, a virtual Linux kernel written from scratch in Zig for lightweight, ephemeral sandboxes. Modal says the hire advances its goal of making Modal Sandboxes a best-in-class product experience.
Physical Intelligence (Pi) runs its Visual-Language-Action robot models on Modal, adding only ~10-15ms of network overhead. Pi worked with Modal to build a QUIC-based portal over UDP with automatic NAT traversal, replacing TCP Tunnels to avoid request-response stalls in the robot control loop. Checkpoints stored in Modal Volumes load in under 30 seconds, and region-pinned deployments keep inference close to robots.
Modal now offers NVIDIA's RTX Pro 6000 Blackwell with 96GB of VRAM and strong fp4/fp8 throughput, available via a single line of code for inference and fine-tuning workloads. The update also adds a dashboard Command Palette (CMD+K), a revamped Sandbox Filesystem API in beta supporting reads up to 5GB and streaming in both directions, and CLI log commands with historical fetching.
Runway is partnering with Modal to power real-time inference for Runway Characters, a real-time video agent API built on Runway's GWM-1 world model that generates expressive digital personas from a single image with zero fine-tuning. Modal's serverless platform let Runway move from proof of concept to production in under 30 days, using single-line multi-node GPU clusters with RDMA networking across every region. Characters is available now to developers and businesses at dev.runwayml.com and to consumers at runwayml.com.
Doppel migrated a large portion of its ML workflows to Modal, removing the infrastructure friction that slowed training experimentation and inference deployment. Parallel experiments via map() replaced sequential runs, while image layer caching and persistent volumes cut build times by up to 10x, with complex setups reaching sub-minute warm builds.
Vapi hosted a webinar in which Principal Product Manager Nick Robin demoed Composer, its in-product assistant for building voice agents without code, producing a working inbound customer service agent within a 30-minute demo. Composer writes system prompts, provisions phone numbers, sets scheduling and flags its own errors for approval. It is currently in alpha and supports both inbound and outbound assistants, CRM and calendar integrations, and editing of existing agents by assistant ID.
Modal added Directory Snapshots for Sandboxes, letting users snapshot a single directory and mount it into a new Sandbox independently of the base image. Modal also released a free public GLM-5 endpoint with Z.ai through the end of April, compatible with OpenCode, Claude Code, OpenClaw and the Vercel AI SDK. The modal billing report CLI and modal.billing.workspace_billing_report API are now generally available for Team and Enterprise workspaces.
Vapi released Vapi Skills, a repo following the Agent Skills standard, plus a Model Context Protocol connector that gives coding agents live access to Vapi's documentation. One install command loads step-by-step instructions with exact API schemas for workflows like create-assistant, create-tool, create-call, and setup-webhook, and the skill pack runs on Claude Code, Cursor, VS Code Copilot, and Gemini CLI.
Modal, a cloud infrastructure platform for data- and compute-intensive AI workloads, highlights research from Hazy Research, Scaling Intelligence, and IRIS that used its GPU fleet to speed up AI research. The KernelBench project, led by Scaling Intelligence's Anne Ouyang, Simon Guo, and Will Hu, provides a 250-problem benchmark and research environment for automated GPU kernel generation, using Modal for consistent, scalable benchmarking across hardware from L40s to Blackwell systems.
Modal launched Directory Snapshots in public beta, letting users snapshot a specific directory inside a running Sandbox and mount it into another Sandbox independently of the base image and the rest of the filesystem. The feature targets coding agents and vibe coding platforms that need to preserve project state across sandbox restarts, and Modal says it enables separating system dependencies from application code, pre-warming sandbox pools, and faster pre-loading of key files. Directory Snapshots persist for 30 days after last use and mount instantly, with contents prioritized for pre-loading on Modal's lazy-loading container filesystem.
Ramp built Ramp Inspect, an internal background coding agent running on Modal Sandboxes, and it now starts roughly half of all merged pull requests across Ramp's frontend and backend repos. Each session boots a full-stack environment with Postgres, Redis, Temporal and RabbitMQ in seconds, because a Modal cron job clones repos and saves filesystem snapshots every 30 minutes. Modal Functions, Dicts and Queues handle snapshot builds, session locks and prompt routing, letting Inspect support multiplayer sessions and hundreds of concurrent sessions without one affecting another.
Vapi shipped Composer, an AI assistant inside the Vapi Dashboard that builds voice agents from a description, writing the prompt, configuring settings, provisioning phone numbers, building tools and setting up integrations. Composer can also pull call logs to explain a failed call and apply a fix, and generate custom code tools that run on Vapi's infrastructure for integrations such as HubSpot. It is available now in Alpha with no setup or onboarding flow.
Modal 推出 GLM-5 的免费 OpenAI API 兼容端点，可用至 4 月底，限单个并发请求，同时发布在单节点 8 张 B200 上用 SGLang 部署的可复现代码。
Why it matters: Modal 为 Z.ai 的 GLM-5 提供免费 OpenAI 兼容端点，并给出在 OpenCode、OpenClaw、Claude Code 和 Vercel AI SDK 中接入的具体配置。
Modal is now purchasable through AWS and Google Cloud Marketplaces, letting enterprise customers apply existing cloud spend commitments to Modal usage. The platform also shipped Sandbox UI updates with clearer resource and region details plus a new execution timeline, and client 1.3.1 adds Python 3.14 support including experimental free-threaded Python (3.14t) in Modal containers.
Factory launched Agent Readiness, which evaluates how well a codebase supports autonomous development across eight technical pillars and five maturity levels, runnable via /readiness-report in Droid, an organization dashboard, or an API. Each criterion is binary and a level unlocks only when 80% of its criteria and all prior levels pass; grounding each evaluation on the repository's previous report cut score variance from an average of 7% to 0.6% across 9 benchmark repositories over six weeks. Reports can trigger an agent that opens a pull request to fix failing criteria such as missing AGENTS.md, linter configuration, or pre-commit hooks.
Chai Discovery is using Modal to run its machine learning drug discovery pipelines, from protein embeddings and multiple sequence alignments to antibody design. Modal Volumes downloads and indexes datasets reaching hundreds of gigabytes once, then shares them across machines, while elastic GPU scaling spins up hundreds of GPUs in minutes. Chai says the same setup now carries research workflows into production without rewriting infrastructure.
OpenAI released a public Model Context Protocol (MCP) server that gives agents read-only access to developer documentation across developers.openai.com, platform.openai.com, and learn.chatgpt.com. The server is documentation-only and does not call the OpenAI API on the user's behalf. Developers can add it to Codex via the CLI or IDE extension with a single shared configuration, and pair it with the OpenAI Docs Skill for citation-backed answers.
Modal has scaled its multi-cloud GPU worker pool past 20,000 concurrent GPUs and launched over four million cloud instances in the last couple years, and is now sharing the reliability system behind it. The post covers instance type testing and selection, machine image preparation and boot checks, passive and active GPU healthchecking, and observability and support. It reports hyperscaler differences such as Cloud A's 99.6% boot success and Cloud C's H100s running above 90ºC in 2025.
Vapi released Vapi Voices Beta, a family of voice models built for high-volume, cost-sensitive voice AI use cases, priced at $0.0025 per minute with latency optimized for conversational responsiveness. The models appear in the existing voice dropdown alongside providers like ElevenLabs, PlayHT and Deepgram, so teams can route high-volume, low-stakes flows such as appointment reminders and routing menus to them with zero integration changes. Vapi says it is gating general availability on what beta usage reveals about use cases, latency under real load and whether the savings matter.
Modal now offers Day 0 support for Mistral 3, letting developers deploy the open model family on serverless GPUs without managing infrastructure. On the 3B Ministral 3 model, Modal's GPU snapshotting feature cut median cold start time from about 118s to about 12s, nearly a 10x reduction. Ministral 3 comes in 3B, 8B, and 14B sizes and performs competitively with the Qwen 3-VL series on benchmarks.
Modal argues that the same developer-experience principles that help humans—tight feedback loops, programmatic access via CLI/SDK/API, actionable error messages, and consistent code-adjacent examples—also make coding agents faster. Because agents iterate by generating, running, and observing code, vague errors or UI-only config stall them, so designing for agents means applying the same principles rather than reinventing them.
Vapi now supports structured outputs for all voice AI assistants, ensuring every call returns data that matches a developer-defined JSON schema. Schemas are enforced at runtime using provider-native capabilities from OpenAI, Anthropic, Gemini and others, and can be reused across multiple assistants. Extracted data arrives in call completion webhooks and the REST API.
Replit launched Fast Mode, a faster way to direct its Agent to make small, precise changes. It runs on an efficient agentic model that is significantly more affordable than Build mode and optimized to finish requests in the fewest steps, aimed at quick fixes, visual iterations and small features. Fast Mode is available to all Replit users starting today and is activated via the lightning bolt icon.
Reducto migrated inference workloads for its 30+ document-processing models to Modal, cutting P90 latency by 3x. GPU memory snapshotting reduced cold boots by 83%, from ~70s to ~12s, while per-customer autoscaling pools isolate traffic spikes. In a load test, Reducto scaled its ingestion pipeline to over 1,000 GPUs in under an hour to demonstrate 100k pages per minute.
Modal's blog explains how host overhead—gaps where the GPU sits idle waiting for the CPU to prepare work—shows up as low GPU kernel utilization in AI inference. At 50% host overhead, GPU inference costs are 2x what they could be. The post details fixes used in production open source inference engines, including constructing tensors on-device instead of transferring them and reducing unnecessary CUDA kernel launches.
Decagon and Modal partnered to build Decagon Voice 2.0, cutting latency by 65% while improving intent recognition and response quality. Decagon fine-tuned compact open-source models with SFT and RL to surpass larger frontier models, while Modal trained a custom EAGLE3 draft model with 38% higher accept lengths and re-engineered SGLang with an async speculative decoding scheduler, boosting throughput up to 12%.
Modal's Ben Shababo walks through building a conversational voice bot on Modal with the open source Pipecat framework and open weights models, reaching a median voice-to-voice latency of about one second when the client and containers are geographically close. The pipeline uses NVIDIA parakeet-tdt-0.6b-v3 for STT, Qwen3-4B-Instruct-2507 served with vLLM, KokoroTTS for speech output, and ChromaDB with all-minilm-l6-v2 for RAG over Modal's docs. Modal Tunnels bypass the input plane for low-latency WebSocket and HTTP links to the STT, TTS and vLLM services, while a FunctionCall tied to each conversation session recovers autoscaling, and pinning services to nearby regions cuts network latency.
Modal has released Volumes v2 in open beta for all apps, with a filesystem rebuilt for higher throughput, faster random access, and true high-concurrency writes from hundreds of containers, plus no cap on stored files. Its JavaScript/TypeScript and Go SDKs also entered beta at v0.5, adding a unified Client object and support for Sandboxes, Functions, Images, and Volumes.
Modal has partnered with Datalab to let developers deploy the Marker document-parsing model and Surya OCR toolkit on Modal GPUs in under five minutes. Marker is a sub-billion-parameter model supporting over 90 languages, and running it on Modal reaches about 2.2 pages per second per container, roughly 10x the 0.22 pages per second on an M4 Mac using Apple MPS. Datalab's tools stay free for research, personal use, and startups under $2M in funding or revenue, with commercial licensing available.
Vapi is giving its community free, unrestricted access to Cartesia's newly launched Sonic-3 TTS model through its managed API key from October 28 through November 2. Sonic-3 adds emotion and laughter tags, fine-grained speed and volume controls, and support for 27 new languages. Vapi recommends starting with the Kira and Ariana voices for low-latency, natural-sounding agents.
Modal engineers read the released Flash Attention 4 source code and reverse-engineered how the Blackwell-optimized attention kernel works, since no official technical report exists yet. The write-up traces a tile through the kernel's warp-specialized asynchronous pipeline and details two math tricks: a cubic-polynomial software exponential that can replace SFU-based exponentiation, and a smarter online softmax rescaling rule that Tri Dao said cuts corrections by a factor of 10. FA4 is reported to be roughly 20% faster than Nvidia's closed-source cudnn attention kernels, and a backwards version was not yet available at the time of writing.
Why it matters: Modal engineers read the released Flash Attention 4 source and explain its warp-specialized pipeline and two math tricks, offering a transferable view of where CUDA kernel design is heading.
Modal built Modal Vibe, an open-source demo vibe coding platform on Modal Sandboxes that scales from 0 to 1,000 AI-coded apps in about five minutes. A load test creating 1,000 apps, each in its own Sandbox, took roughly 300 seconds, with the AI API as the rate-limiting step. Modal says the Team Plan can run 1,000 concurrent Sandboxes, enough for over one million sessions per month.
Modal shipped product updates including Modal Notebooks for cloud-based Python experimentation with up to 8 B200 GPUs, plus an optional idle_timeout parameter for modal.Sandbox.create(). The client also added a startup_timeout parameter to @app.function() and @app.cls() decorators and a new API pattern for imperative management of Modal resource types. Modal Notebooks have since been shut down, though existing notebooks can still be downloaded as .ipynb files.
Modal detailed the engineering behind Modal Notebooks, its cloud Jupyter notebook that boots GPUs and custom images in seconds with real-time collaboration. Kernels run inside Modal Sandboxes, with a modal-kernelshim daemon translating Jupyter ZeroMQ messages into HTTP calls tunneled through Modal's control plane. A lazy-loading Rust FUSE container filesystem fetches file contents on demand through a tiered cache, avoiding the image-unpacking latency that can approach a minute for an 8 GB Python/ML image.
Modal Notebooks is now generally available to all Modal customers after a limited beta last month. The collaborative notebook environment runs on a GPU-enabled Python kernel that reaches a ready state in under 5 seconds on arbitrary container images and hardware up to 256 vCPUs and 8 H100/B200 GPUs, with auto-idle shutdown, shared Volumes and Secrets, and real-time multi-cursor editing. Modal says over 5,000 accounts ran 200,000 code cells during the August beta, and upcoming features include memory snapshots, one-click export to Modal Apps, and scheduled runs.
Zencastr moved its podcast transcription workloads to Modal's GPU platform, scaling to 1,500 concurrent GPUs within minutes to process hundreds of years of audio in just a few days. The shift from self-managed Kubernetes GPU nodes to Modal's scale-to-zero infrastructure cut idle GPU costs and removed driver and dependency management overhead. All core ML workloads now run on Modal, including transcription, speaker counting, laughter detection, and audio quality scoring.
Modal added GPU memory snapshots in alpha, cutting cold boots for GPU workloads by 10x by skipping steps like Torch compilation, enabled via the enable_gpu_snapshot option. The update also brings Modal Notebooks in beta for running Python in the browser, Service Tokens for workspace managers, and browser-generated UIs for deployed Functions.
Modal announced GPU memory snapshots in alpha, extending its existing checkpoint/restore system to capture GPU vRAM, CUDA kernels, streams and contexts so GPU workloads restore from a frozen state instead of rebuilding CUDA state after boot.
Why it matters: Modal adds GPU state to its memory snapshots, cutting cold boots for torch.compile and vLLM workloads by up to 10x and making scale-to-zero more practical.
An AI code sandbox is an isolated environment that executes LLM-generated code at runtime, containing the blast radius of errors or injected vulnerabilities. Modern implementations run on VMs or containers, often using secure runtimes like gVisor to intercept syscalls and shield the host. Most companies choose managed sandbox services over building their own with Kubernetes, since cold starts and scaling to millions of concurrent sessions are costly to handle.