Vapi released the Vapi MCP Server, letting AI agent tools trigger real-time voice flows and outbound calls via function calling. It supports Claude Desktop integration and remote access over SSE, so users can ask Claude to place calls or schedule follow-ups in plain language. The server is available via @vapi-ai/mcp-server, locally or through a hosted SSE endpoint.
Modal now lets each Function queue up to 1 million inputs, up from 2k, with higher .spawn() rate limits and FunctionCall results retained for 7 days. The update also adds ephemeral apps via app.run() inside containers, Git commit info for apps in the CLI and dashboard, and a new TensorRT-LLM example serving LLMs at under 400 ms latency.
Modal has unveiled a redesigned logo and wordmark alongside its first out-of-home brand campaign, with three billboards running across San Francisco until the end of May. The new logo, developed with Ty Wilkins, replaces the thin-lined cube "M" to stay legible from favicons to billboards. The campaign highlights Modal as an AI infrastructure platform, and the NYC-based company chose SF to reach the tech and AI epicenter.
Consultant Christian Ulstrup of GSD at Work used Vapi voice AI agents to replace scheduled discovery calls with an asynchronous voice intake system for his AI Action Sessions workshops. Participants call a dedicated Vapi phone line at their convenience, and transcripts plus AI-generated summaries flow into Slack via Zapier and are logged in HubSpot. For one recent client, 20 participants across the globe contributed over 3 hours of discovery, and the session earned a 5 out of 5 rating.
Modal cut GPU, CPU and memory prices by up to 65%, and Team plan users can now run up to 50 GPUs concurrently. The Modal SDK is now on conda-forge starting with version 0.73.46, and a version 1.0 client release is planned. Modal also published an LTX-Video image-to-video example and launched a new logo.
AI call centers are reshaping customer support by resolving routine calls in seconds instead of long waits, as in the example of an outage query handled in 30 seconds. Gartner estimates about 25% of customer service interactions will involve AI by 2027, up from 2% in 2022, a market worth $15-20 billion by 2027-2028 at roughly 25-30% CAGR. The article covers traditional call center pain points such as slow response times, peak-hour surges and limited hours, and how AI automation addresses them.
Modal published a guide distinguishing three things commonly called "GPU utilization": GPU Allocation Utilization (GPU-seconds running application code over GPU-seconds paid for), GPU Kernel Utilization (fraction of time running code on GPUs), and Model FLOP/s Utilization (share of theoretical arithmetic bandwidth used). The guide focuses on neural network inference workloads, which Modal calls a revenue center rather than a cost center.
Modal has open sourced its GPU Glossary, an interlinked compendium of short documents on GPU programming topics, on GitHub under a Creative Commons BY 4.0 License. The glossary was first released two months ago to fill a gap in cross-stack GPU reference material, and community requests to contribute prompted the move. The repo's GitHub Issues already list planned extensions, including two sample matmul kernels and a section on Thread Block Clusters.
Modal shipped a refreshed logs dashboard with time-range input filtering, live container profiling for debugging stuck containers or inputs, and Debug Shells that run commands like vim, ps, and strace inside running containers. Region selection is now available on all plans, priced with a multiplier on base prices, and the client adds OIDC authentication for cloud bucket mounts via the new oidc_auth_role_arn field.
Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.
Why it matters: Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.
Modal now offers NVIDIA L40S GPUs at $1.95/hr, with 48GB of memory, up to 40% faster performance on memory-bound tasks and over 100% speedup on compute-bound jobs versus its A10 GPUs. The platform also added Proxy Auth tokens for gating web endpoints, a Filesystem API for Sandboxes, and disk snapshotting that lets users branch new Sandbox instances from snapshots to eliminate cold starts. Modal additionally announced completion of its SOC 2 Type 2 certification.
Modal announced the general availability of Sandboxes, its primitive for safely running untrusted code from LLMs, users, or third-party sources in an isolated environment separate from other Modal resources. Sandboxes offer a simple exec API, runtime-configurable environments via the Image API, and filesystem snapshots for persistence. Modal also upstreamed support into SWE-bench, where a --modal flag lets the Verified benchmark's 500 tasks run in about 7 minutes, and cited Quora's Poe, Codegen, and Relevance AI as customers.
Modal shipped a product update round led by a major upgrade to its beta memory snapshotting feature, which it says can cut cold start times by half for models like Stable Diffusion. The platform also added OpenID Connect support for short-lived identity tokens, eStargz image compression for faster builds from registries like ECR and Docker Hub, and is expanding async job queues toward a 1 million input limit, up from 2k.
Modal now offers NVIDIA L40S GPUs to all users at $1.95/hr, positioned as a faster, larger-memory alternative to its most popular inference accelerator, the A10. The L40S doubles GPU RAM to 48GB, letting models like Flux.1-schnell run in 16bit precision on a single GPU, and delivers roughly 40% speedup on memory-bound jobs and over 100% on compute-bound 16bit Tensor Core jobs.
Fine-tuning adjusts pretrained LLM weights to fit a specific task or domain, delivering higher quality than prompt engineering at lower cost and latency; a fine-tuned Llama 7B can be roughly 50 times more cost-effective per token than GPT-3.5 with comparable performance. The guide covers use cases such as structured output, style adherence and domain-specific instruction, plus options including OpenAI, Predibase, Modal, Google Colab and AWS SageMaker, and frameworks HuggingFace transformers, TRL and Axolotl.
Modal held an internal hackathon where 30 employees formed 13 teams to build projects that all used Modal. Browserman is a multi-modal AI agent built with Llama-3.2-90B-Vision-Instruct-FP8, vLLM and Playwright that navigates the web visually to complete tasks. Glodal prototyped a distributed control plane that cut request latency from 268ms to 3ms, and Modal is now working to build it into the platform.
A top-tier European soccer team, anonymized as AFC Richmond, moved its match tracking data pipeline to Modal's serverless GPU platform and cut the cost of processing a full season of games by 50%. The team runs inference on ~3.5 million tracking observations per game through a custom transformer-based model, and with Airflow on Modal now processes each game's data in minutes rather than hours.
Modal signed a Strategic Collaboration Agreement with AWS to help businesses develop and deploy AI products faster on its serverless platform for GPU-accelerated workloads, which can spin up GPU-enabled containers in as little as one second. As part of the deal, Modal will list on AWS Marketplace and invest in integrations such as AWS PrivateLink to serve customers with specific security and privacy requirements.
OpenArt deployed its proprietary ComfyUI workflows on Modal, cutting deployment from hours on AWS and GCP to minutes and autoscaling to hundreds of GPU containers. Modal now powers 100+ workflows on the platform, which serves over 3 million monthly active users.
Cursor announced that Supermaven, the fast context-aware AI coding assistant, is joining the company, with Supermaven creator Jacob Jackson and his team coming aboard. Cursor said it plans a new Tab model version that is fast, context-aware and especially good at handling long chains of large edits, while the Supermaven plugin will keep being maintained even as Cursor becomes the team's core focus.
Modal shipped static IP proxies that route outbound traffic through workspace-exclusive IPs, plus a native Slack integration for real-time app alerts and a live usage dashboard tracking resources across all workspace apps. The update also adds GPU fallbacks for functions, expanding the pool of resources a function can be scheduled on, and client changes including `modal shell` container IDs and memory snapshotting for parametrized functions.
Tidbyt, the NYC-based maker of programmable smart home LED screens, is joining Modal. Tidbyt co-founders Rohan Singh and Mats Linander, who previously worked on container infrastructure at Spotify, bring large-scale systems experience to Modal's cloud infrastructure for data/AI/ML teams. Tidbyt's service will keep running for existing devices, but no new devices will ship.
Modal argues that AI's GPU crunch stems from a mismatch: most demand goes to training, but inference is where revenue comes from, and inference workloads are inherently volatile. Because request rates swing with day-night cycles, weekly patterns, viral traffic and growth trends, fixed long-term GPU reservations push real-world cluster utilization below 50%. Pooling demand across many users — provisioning for the peak of the sum rather than the sum of peaks — can raise utilization sharply, and pooling supply across regions can add further flexibility.
A demo app performs hybrid search over the state of California using timestamps, geolocation data, and satellite image embeddings from the Clay v1 foundation model. Modal orchestrates daily Sentinel API queries, MongoDB Atlas storage, and GPU embedding inference on A10 GPUs, with a small Alpine JS frontend served via FastAPI.
Contextual AI automated and parallelized its GPU-based continuous integration on Modal, replacing manual test runs on in-house GPU nodes. A GitHub Action triggers a Modal Function with multiple GPUs attached, and cached image builds let tests start within 30 seconds of job submission. Modal bills by usage, so 99% of what is billed is actual test run-time.
Modal has added native dynamic batching support, letting inference functions group requests via a single @modal.batched() decorator with max_batch_size and wait_ms parameters. On OpenAI's Whisper large v3 running on an A10G, a max_batch_size of 64 raised throughput from ~1.2 to ~3.3 requests per second per container and cut inference cost by 65%.
Modal shipped app rollbacks that let users revert a deployed app to a previous version from the dashboard or CLI, plus a @batched decorator that accumulates requests into dynamically sized batches to trade latency for throughput and GPU utilization. Sandbox objects can now forward TCP ports to the internet and exec additional processes, and logs can be exported to an OpenTelemetry provider. CPU cores and higher-end GPUs also got cheaper.
Modal now supports HIPAA-compliant workloads, available to Enterprise plan customers who sign a Business Associate Agreement (BAA) via security@modal.com. All Modal features fall within the BAA's scope except Volumes, Images (excluding Filesystem and Directory Snapshots), Memory Snapshots, and user code, where PHI should not be placed. Modal cites its least-privilege data handling and gVisor sandboxing runtime as security foundations, building on its SOC 2 Type 1 announcement and ongoing SOC 2 Type 2 certification.
Cursor has implemented a shadow workspace that lets its AI apply edits in a hidden Electron window and read back lint results without touching the user's editing session. The post outlines the design goals of LSP availability, independence, privacy, concurrency and speed, and explains why copying a TextModel or running separate language servers failed. It also sketches a future kernel-level folder proxy for runnability, noting that macOS and Windows lack a viable FUSE path and that Rust LSP support is currently unavailable.
Cursor raised a $60M Series A from Andreessen Horowitz, Thrive Capital, OpenAI, Jeff Dean, Noam Brown, and founders of Stripe, GitHub, Ramp, Perplexity and OpenAI. The AI coding tool says it has grown to more than 40,000 users, including startups, research institutions and large enterprises, and describes internal systems for next-edit prediction, retrieval across billions of files and speculative-inference code rewriting.
Why it matters: Cursor's $60M Series A announcement also discloses its user base has grown past 40,000 and names the investors backing its AI coding tool.
Modal is cutting prices on CPUs and some of its most powerful GPUs by 15-30%, citing increased competition in the GPU supply pool and the launch of more powerful GPUs over the past year. The company is also changing CPU/memory billing to charge max(reservation, usage) per running container, with a minimum default reservation of 0.125 CPU cores for all Functions. The Team plan rises to $250/mo with unlimited seats and $100/mo in credits, with new pricing taking effect in September for affected Starter and Team/Enterprise users.
Modal's Charles Frye replicated the core claim of the Large Language Monkeys paper on a new dataset, showing that repeated sampling lifts LLaMA 3.1 8B from 66.4% pass@1 to 90.5% pass@100 and 95.1% pass@1000 on HumanEval, versus GPT-4o's reported 90.2% pass@1. The runs used vLLM on Modal serverless GPUs and Modal Sandboxes for parallel test execution, cost well under $50, and showed smooth, predictable scaling across three orders of magnitude.
Why it matters: A hands-on replication shows how repeated sampling with a small open model can approach GPT-4o on HumanEval, offering a concrete, low-cost recipe for trading compute for accuracy.
Modal's monthly product update adds a Datadog integration that forwards application and audit logs with a single click, plus cloud region selection for functions to meet regulatory, egress cost, or database-locality needs. The company also reworked its function call system for higher throughput with significantly lower overhead, targeting low-latency real-time production workloads, and overhauled the app details page with time-interval zoom, container status, and image file system views.
Basis organized the Prompt Olympics, a competitive programming-style prompt engineering contest with a $5,000 prize, and ran it on Modal's infrastructure. In one challenge, participants wrote prompts that steered LLaMA 3 70B, deployed on Modal, to generate code that had to pass tests inside Modal Sandboxes. The finale had players prompt a model to control the Crafter game agent; Basis found even capable open source models like LLaMA 3 70B hard to prompt effectively and switched to a proprietary model API for that component.
Modal can now be triggered directly from an Airflow DAG, letting teams run GPU-backed AI/ML tasks without setting up Celery or Kubernetes executors. The guide covers installing Modal in an Airflow environment, setting MODAL_TOKEN_ID and MODAL_TOKEN_SECRET, and two approaches: calling deployed Modal Functions via lookup, or building a custom operator that runs Python in Modal Sandboxes. A Modal Airflow provider package is in progress.
Modal added a runtime program-analysis component called seccheck to its serverless container runtime, detecting and disabling cryptomining Functions before banning the offending users. Because miners can change metadata but not their programs, Modal analyzes syscall logs rather than source code, binaries, or container images. The system avoids ptrace-based strace overhead, which can slow syscall-heavy programs by over 100x.
Cursor published a follow-up to its original "Problems" post, listing the research questions it considers most critical for the next phase of AI coding. The company has extended Copilot++ to predict the next cursor position, and combining that with next-edit prediction lets the model "play" sequences of low-entropy changes—a demo required 11 Tab presses versus 3 presses of every other key, which Cursor calls Cursor Flow. Other directions include action prediction across the codebase, perfect edits via more inference-time compute, hallucinated pseudocode, multi-file editing, optimal context, multi-hop context, and bug detection and debugging.
Hunch, a spatial canvas that combines models such as OpenAI GPT-4o, Anthropic Claude 3 Opus and Stable Diffusion 3, uses Modal Sandboxes to execute AI-generated code safely in isolated, ephemeral containers. The integration required only a few lines of Python and supports pip installs, attached storage volumes and automatic scaling. Hunch said the setup lets non-coders build workflows that fetch API data or send results to Slack webhooks.
Modal fine-tuned a Stable Diffusion model on the roughly 300-SVG Heroicons library to generate new black-and-white line icons in that style, including ones for concepts like barack-obama and golden-retriever. The full pipeline — downloading the pretrained model, full fine-tuning with HuggingFace Diffusers' train_text_to_image.py script, and serving a Gradio web UI — runs on Modal's serverless GPU platform. The write-up walks through dataset prep (SVG-to-PNG conversion, white backgrounds, caption metadata.csv) and notes that full fine-tuning worked best, though LoRA-style parallel adapters can also work for small datasets.
Substack has moved training and deployment for nearly all of its ML models—including spam detection, newsletter recommendations, audio transcription, sentiment analysis and image generation—from AWS SageMaker to Modal. Head of AI & ML Engineering Mike Cohen cited SageMaker's convoluted developer experience, hard-to-share code across notebooks and 5+ minute container startup times; on Modal, an inference endpoint took an hour to implement. Substack now runs a full fine-tune, validation and deployment workflow on Modal's storage primitives.