Modal held an internal hackathon where 30 employees formed 13 teams to build projects that all used Modal. Browserman is a multi-modal AI agent built with Llama-3.2-90B-Vision-Instruct-FP8, vLLM and Playwright that navigates the web visually to complete tasks. Glodal prototyped a distributed control plane that cut request latency from 268ms to 3ms, and Modal is now working to build it into the platform.
A top-tier European soccer team, anonymized as AFC Richmond, moved its match tracking data pipeline to Modal's serverless GPU platform and cut the cost of processing a full season of games by 50%. The team runs inference on ~3.5 million tracking observations per game through a custom transformer-based model, and with Airflow on Modal now processes each game's data in minutes rather than hours.
Modal signed a Strategic Collaboration Agreement with AWS to help businesses develop and deploy AI products faster on its serverless platform for GPU-accelerated workloads, which can spin up GPU-enabled containers in as little as one second. As part of the deal, Modal will list on AWS Marketplace and invest in integrations such as AWS PrivateLink to serve customers with specific security and privacy requirements.
OpenArt deployed its proprietary ComfyUI workflows on Modal, cutting deployment from hours on AWS and GCP to minutes and autoscaling to hundreds of GPU containers. Modal now powers 100+ workflows on the platform, which serves over 3 million monthly active users.
Cursor announced that Supermaven, the fast context-aware AI coding assistant, is joining the company, with Supermaven creator Jacob Jackson and his team coming aboard. Cursor said it plans a new Tab model version that is fast, context-aware and especially good at handling long chains of large edits, while the Supermaven plugin will keep being maintained even as Cursor becomes the team's core focus.
Modal shipped static IP proxies that route outbound traffic through workspace-exclusive IPs, plus a native Slack integration for real-time app alerts and a live usage dashboard tracking resources across all workspace apps. The update also adds GPU fallbacks for functions, expanding the pool of resources a function can be scheduled on, and client changes including `modal shell` container IDs and memory snapshotting for parametrized functions.
Tidbyt, the NYC-based maker of programmable smart home LED screens, is joining Modal. Tidbyt co-founders Rohan Singh and Mats Linander, who previously worked on container infrastructure at Spotify, bring large-scale systems experience to Modal's cloud infrastructure for data/AI/ML teams. Tidbyt's service will keep running for existing devices, but no new devices will ship.
Modal argues that AI's GPU crunch stems from a mismatch: most demand goes to training, but inference is where revenue comes from, and inference workloads are inherently volatile. Because request rates swing with day-night cycles, weekly patterns, viral traffic and growth trends, fixed long-term GPU reservations push real-world cluster utilization below 50%. Pooling demand across many users — provisioning for the peak of the sum rather than the sum of peaks — can raise utilization sharply, and pooling supply across regions can add further flexibility.
A demo app performs hybrid search over the state of California using timestamps, geolocation data, and satellite image embeddings from the Clay v1 foundation model. Modal orchestrates daily Sentinel API queries, MongoDB Atlas storage, and GPU embedding inference on A10 GPUs, with a small Alpine JS frontend served via FastAPI.
Contextual AI automated and parallelized its GPU-based continuous integration on Modal, replacing manual test runs on in-house GPU nodes. A GitHub Action triggers a Modal Function with multiple GPUs attached, and cached image builds let tests start within 30 seconds of job submission. Modal bills by usage, so 99% of what is billed is actual test run-time.
Modal has added native dynamic batching support, letting inference functions group requests via a single @modal.batched() decorator with max_batch_size and wait_ms parameters. On OpenAI's Whisper large v3 running on an A10G, a max_batch_size of 64 raised throughput from ~1.2 to ~3.3 requests per second per container and cut inference cost by 65%.
Modal shipped app rollbacks that let users revert a deployed app to a previous version from the dashboard or CLI, plus a @batched decorator that accumulates requests into dynamically sized batches to trade latency for throughput and GPU utilization. Sandbox objects can now forward TCP ports to the internet and exec additional processes, and logs can be exported to an OpenTelemetry provider. CPU cores and higher-end GPUs also got cheaper.
Modal now supports HIPAA-compliant workloads, available to Enterprise plan customers who sign a Business Associate Agreement (BAA) via security@modal.com. All Modal features fall within the BAA's scope except Volumes, Images (excluding Filesystem and Directory Snapshots), Memory Snapshots, and user code, where PHI should not be placed. Modal cites its least-privilege data handling and gVisor sandboxing runtime as security foundations, building on its SOC 2 Type 1 announcement and ongoing SOC 2 Type 2 certification.
Cursor has implemented a shadow workspace that lets its AI apply edits in a hidden Electron window and read back lint results without touching the user's editing session. The post outlines the design goals of LSP availability, independence, privacy, concurrency and speed, and explains why copying a TextModel or running separate language servers failed. It also sketches a future kernel-level folder proxy for runnability, noting that macOS and Windows lack a viable FUSE path and that Rust LSP support is currently unavailable.
Cursor raised a $60M Series A from Andreessen Horowitz, Thrive Capital, OpenAI, Jeff Dean, Noam Brown, and founders of Stripe, GitHub, Ramp, Perplexity and OpenAI. The AI coding tool says it has grown to more than 40,000 users, including startups, research institutions and large enterprises, and describes internal systems for next-edit prediction, retrieval across billions of files and speculative-inference code rewriting.
Why it matters: Cursor's $60M Series A announcement also discloses its user base has grown past 40,000 and names the investors backing its AI coding tool.
Modal is cutting prices on CPUs and some of its most powerful GPUs by 15-30%, citing increased competition in the GPU supply pool and the launch of more powerful GPUs over the past year. The company is also changing CPU/memory billing to charge max(reservation, usage) per running container, with a minimum default reservation of 0.125 CPU cores for all Functions. The Team plan rises to $250/mo with unlimited seats and $100/mo in credits, with new pricing taking effect in September for affected Starter and Team/Enterprise users.
Modal's Charles Frye replicated the core claim of the Large Language Monkeys paper on a new dataset, showing that repeated sampling lifts LLaMA 3.1 8B from 66.4% pass@1 to 90.5% pass@100 and 95.1% pass@1000 on HumanEval, versus GPT-4o's reported 90.2% pass@1. The runs used vLLM on Modal serverless GPUs and Modal Sandboxes for parallel test execution, cost well under $50, and showed smooth, predictable scaling across three orders of magnitude.
Why it matters: A hands-on replication shows how repeated sampling with a small open model can approach GPT-4o on HumanEval, offering a concrete, low-cost recipe for trading compute for accuracy.
Modal's monthly product update adds a Datadog integration that forwards application and audit logs with a single click, plus cloud region selection for functions to meet regulatory, egress cost, or database-locality needs. The company also reworked its function call system for higher throughput with significantly lower overhead, targeting low-latency real-time production workloads, and overhauled the app details page with time-interval zoom, container status, and image file system views.
Basis organized the Prompt Olympics, a competitive programming-style prompt engineering contest with a $5,000 prize, and ran it on Modal's infrastructure. In one challenge, participants wrote prompts that steered LLaMA 3 70B, deployed on Modal, to generate code that had to pass tests inside Modal Sandboxes. The finale had players prompt a model to control the Crafter game agent; Basis found even capable open source models like LLaMA 3 70B hard to prompt effectively and switched to a proprietary model API for that component.
Modal can now be triggered directly from an Airflow DAG, letting teams run GPU-backed AI/ML tasks without setting up Celery or Kubernetes executors. The guide covers installing Modal in an Airflow environment, setting MODAL_TOKEN_ID and MODAL_TOKEN_SECRET, and two approaches: calling deployed Modal Functions via lookup, or building a custom operator that runs Python in Modal Sandboxes. A Modal Airflow provider package is in progress.
Modal added a runtime program-analysis component called seccheck to its serverless container runtime, detecting and disabling cryptomining Functions before banning the offending users. Because miners can change metadata but not their programs, Modal analyzes syscall logs rather than source code, binaries, or container images. The system avoids ptrace-based strace overhead, which can slow syscall-heavy programs by over 100x.
Cursor published a follow-up to its original "Problems" post, listing the research questions it considers most critical for the next phase of AI coding. The company has extended Copilot++ to predict the next cursor position, and combining that with next-edit prediction lets the model "play" sequences of low-entropy changes—a demo required 11 Tab presses versus 3 presses of every other key, which Cursor calls Cursor Flow. Other directions include action prediction across the codebase, perfect edits via more inference-time compute, hallucinated pseudocode, multi-file editing, optimal context, multi-hop context, and bug detection and debugging.
Hunch, a spatial canvas that combines models such as OpenAI GPT-4o, Anthropic Claude 3 Opus and Stable Diffusion 3, uses Modal Sandboxes to execute AI-generated code safely in isolated, ephemeral containers. The integration required only a few lines of Python and supports pip installs, attached storage volumes and automatic scaling. Hunch said the setup lets non-coders build workflows that fetch API data or send results to Slack webhooks.
Modal fine-tuned a Stable Diffusion model on the roughly 300-SVG Heroicons library to generate new black-and-white line icons in that style, including ones for concepts like barack-obama and golden-retriever. The full pipeline — downloading the pretrained model, full fine-tuning with HuggingFace Diffusers' train_text_to_image.py script, and serving a Gradio web UI — runs on Modal's serverless GPU platform. The write-up walks through dataset prep (SVG-to-PNG conversion, white backgrounds, caption metadata.csv) and notes that full fine-tuning worked best, though LoRA-style parallel adapters can also work for small datasets.
Substack has moved training and deployment for nearly all of its ML models—including spam detection, newsletter recommendations, audio transcription, sentiment analysis and image generation—from AWS SageMaker to Modal. Head of AI & ML Engineering Mike Cohen cited SageMaker's convoluted developer experience, hard-to-share code across notebooks and 5+ minute container startup times; on Modal, an inference endpoint took an hour to implement. Substack now runs a full fine-tune, validation and deployment workflow on Modal's storage primitives.
Modal's monthly product update adds writable mounts for AWS S3 and Cloudflare R2 buckets, a @web_server decorator for hosting non-ASGI/WSGI web servers on a specified port, and Okta single sign-on for enterprise plan users. A volume viewer in the web dashboard lets users explore volume file systems, and the latest client adds multi-file app composition via .include, partitioned queues, and stdin support for Sandboxes.
A fine-tuned open source embedding model can outperform OpenAI's text-embedding-3-small in accuracy at lower cost using only a handful of examples. Modal ran an experiment on the Quora duplicate-questions dataset to find how much fine-tuning data a simple question-answering use case needs, and describes how to build one on its serverless infrastructure.
Modal, the serverless platform for AI and data teams, has been named to Wing's sixth annual Enterprise Tech 30 list, ranking No. 3 among ten companies in the early-stage category. The list was selected from nearly 600 venture-backed private enterprise tech companies by 112 venture capitalists and 36 corporate development teams.
Ramp used Modal to fine-tune its LLMs and build a text-to-structured-JSON model for receipt management, cutting receipts requiring manual intervention by 34% on infrastructure estimated 79% cheaper than major LLM providers like OpenAI. Modal functions trained many candidate models in parallel, persisted weights to Modal volumes, and served a parametrized inference endpoint. One engineer also ported a PII-stripping script for 25,000 invoices to Modal, parallelizing it on 256 cloud workers to finish in 20 minutes for $100 instead of 3 days locally.
Modal built modal-http, an HTTP and WebSocket stack that lets its serverless functions accept web requests, with full WebSocket support recently completed. Unlike AWS Lambda's 15-minute and 50 MB limits, Modal containers can use up to 64 CPUs, 336 GB of memory and 8 Nvidia H100 GPUs, so the team translates HTTP into function calls rather than fronting containers with a conventional reverse proxy.
Modal now supports WebSocket connections, letting functions on its platform maintain persistent, bidirectional links between client and server instead of per-request HTTP connections. Each WebSocket connection counts as a single input, so functions not bound by CPU/GPU should allow concurrent inputs or Modal spins up a new container per connection. Modal auto-scales WebSocket handlers, targeting real-time streaming, progress updates on long-running tasks, and hosting frameworks like ComfyUI, Streamlit and Gradio.
Suno used Modal to scale inference and batch pre-processing across thousands of GPUs, bringing its music generation model to market four months early instead of hiring engineers to build and maintain infrastructure. The team ran batch pre-processing with a few short Python scripts and no config files, then expanded to model deployment with HTTP-exposed functions and chained inference across multiple models and containers. Modal's auto-scaling to thousands of GPUs handled holiday demand spikes without long-term GPU reservations.
Modal has added WebSocket support to its functions, enabling out-of-the-box hosting of frameworks like Streamlit, plus interactive commands via `modal container exec` for inspecting and debugging active containers. H100 GPUs are now available on Modal, and the client update adds `@build` lifecycle hooks, automatic background commits for Volumes, and expanded CLI commands including get, put and rm.
Modal has made NVIDIA H100 GPUs available to all users, priced at $7.65 per hour per GPU. NVIDIA benchmarks show the H100 delivers up to 4x training speedups and up to 30x inference speedups versus the A100, with 80 GB of on-chip DRAM and up to 8 GPUs linked via NVLink at 3.6 TB/s. Users enable it by setting "H100" as the GPU type in the Modal decorator.
Modal and Hugging Face's Text Embedding Inference server can embed the entire English Wikipedia in about 15 minutes for just over $15, using Modal's serverless GPU scaling. The post argues open-source embedding models beat API-based ones like OpenAI's text-embedding-ada-002 for production RAG, since they can be fine-tuned on user data and avoid rate limits and per-token costs.
Cursor published a list of problems it is working on, including training a custom fast reranker that filters 500k tokens from all sources down to the most relevant 8k per request. The list also covers an edit-oriented copilot, constrained in-flow agents for large codebases, bug finding, and larger-scale edits. As of Oct 12, 2023, Cursor had indexed 1.4B vectors and 150k codebases.
Modal announced that its cloud function platform is now generally available, letting anyone sign up and run Python code in the cloud within minutes, after opening a waitlist in 2022. The platform offers serverless execution billed per second, autoscaling GPU inference endpoints on A100, A10G, T4, L4 and H100, and large-scale batch jobs across thousands of containers with no YAML configuration. Modal also raised a $16M Series A led by Redpoint Ventures, with Amplify Partners, which led the seed round, and plans to use the capital to accelerate its product roadmap.
Modal Labs has raised a $16 million Series A led by Redpoint Ventures, with existing investor Amplify Partners and new participants including Lux Capital and Definition Capital, bringing total funding to $23 million. The company also officially launched its cloud platform, previously in beta with early customers such as Ramp, Substack and SphinxBio, and will use the funds to accelerate product development and hire engineers at its New York City headquarters.
Cursor measured Llama-2-70B inference on 2 80-GB A100 GPUs and found it over 3x cheaper than GPT-3.5 for prompt tokens but far more expensive for generated tokens at small batch sizes. Prompt processing costs about $0.00042 per 1K tokens versus GPT-3.5's $0.0015, while generation reaches $0.066 per 1K tokens at batch size 1 and only approaches GPT-3.5 pricing at large batches that push time to first token to several seconds. Cursor recommends open-source models for prompt-heavy tasks like classification and reranking, and closed-source APIs for generation-heavy workloads.
Why it matters: Cursor's measured cost and latency data shows when self-hosting Llama-2-70B beats GPT-3.5 and when it does not, offering a reusable framework for open-source model serving decisions.
Cursor researcher Arvid Lunnemark argues that prompting should be called "prompt design" and compared to web design, since both pursue clear communication with dynamic inputs and must fit content to different "sizes" — screen sizes for the web, context windows for prompts. He released Priompt v0.1, a React-like JSX-based prompt design library already used inside Cursor, with a preview site for rendering prompts against real request data.