Modal can now be triggered directly from an Airflow DAG, letting teams run GPU-backed AI/ML tasks without setting up Celery or Kubernetes executors. The guide covers installing Modal in an Airflow environment, setting MODAL_TOKEN_ID and MODAL_TOKEN_SECRET, and two approaches: calling deployed Modal Functions via lookup, or building a custom operator that runs Python in Modal Sandboxes. A Modal Airflow provider package is in progress.
Modal added a runtime program-analysis component called seccheck to its serverless container runtime, detecting and disabling cryptomining Functions before banning the offending users. Because miners can change metadata but not their programs, Modal analyzes syscall logs rather than source code, binaries, or container images. The system avoids ptrace-based strace overhead, which can slow syscall-heavy programs by over 100x.
Cursor published a follow-up to its original "Problems" post, listing the research questions it considers most critical for the next phase of AI coding. The company has extended Copilot++ to predict the next cursor position, and combining that with next-edit prediction lets the model "play" sequences of low-entropy changes—a demo required 11 Tab presses versus 3 presses of every other key, which Cursor calls Cursor Flow. Other directions include action prediction across the codebase, perfect edits via more inference-time compute, hallucinated pseudocode, multi-file editing, optimal context, multi-hop context, and bug detection and debugging.
Hunch, a spatial canvas that combines models such as OpenAI GPT-4o, Anthropic Claude 3 Opus and Stable Diffusion 3, uses Modal Sandboxes to execute AI-generated code safely in isolated, ephemeral containers. The integration required only a few lines of Python and supports pip installs, attached storage volumes and automatic scaling. Hunch said the setup lets non-coders build workflows that fetch API data or send results to Slack webhooks.
Modal fine-tuned a Stable Diffusion model on the roughly 300-SVG Heroicons library to generate new black-and-white line icons in that style, including ones for concepts like barack-obama and golden-retriever. The full pipeline — downloading the pretrained model, full fine-tuning with HuggingFace Diffusers' train_text_to_image.py script, and serving a Gradio web UI — runs on Modal's serverless GPU platform. The write-up walks through dataset prep (SVG-to-PNG conversion, white backgrounds, caption metadata.csv) and notes that full fine-tuning worked best, though LoRA-style parallel adapters can also work for small datasets.
Substack has moved training and deployment for nearly all of its ML models—including spam detection, newsletter recommendations, audio transcription, sentiment analysis and image generation—from AWS SageMaker to Modal. Head of AI & ML Engineering Mike Cohen cited SageMaker's convoluted developer experience, hard-to-share code across notebooks and 5+ minute container startup times; on Modal, an inference endpoint took an hour to implement. Substack now runs a full fine-tune, validation and deployment workflow on Modal's storage primitives.
Modal's monthly product update adds writable mounts for AWS S3 and Cloudflare R2 buckets, a @web_server decorator for hosting non-ASGI/WSGI web servers on a specified port, and Okta single sign-on for enterprise plan users. A volume viewer in the web dashboard lets users explore volume file systems, and the latest client adds multi-file app composition via .include, partitioned queues, and stdin support for Sandboxes.
A fine-tuned open source embedding model can outperform OpenAI's text-embedding-3-small in accuracy at lower cost using only a handful of examples. Modal ran an experiment on the Quora duplicate-questions dataset to find how much fine-tuning data a simple question-answering use case needs, and describes how to build one on its serverless infrastructure.
Ramp used Modal to fine-tune its LLMs and build a text-to-structured-JSON model for receipt management, cutting receipts requiring manual intervention by 34% on infrastructure estimated 79% cheaper than major LLM providers like OpenAI. Modal functions trained many candidate models in parallel, persisted weights to Modal volumes, and served a parametrized inference endpoint. One engineer also ported a PII-stripping script for 25,000 invoices to Modal, parallelizing it on 256 cloud workers to finish in 20 minutes for $100 instead of 3 days locally.
Modal built modal-http, an HTTP and WebSocket stack that lets its serverless functions accept web requests, with full WebSocket support recently completed. Unlike AWS Lambda's 15-minute and 50 MB limits, Modal containers can use up to 64 CPUs, 336 GB of memory and 8 Nvidia H100 GPUs, so the team translates HTTP into function calls rather than fronting containers with a conventional reverse proxy.
Modal now supports WebSocket connections, letting functions on its platform maintain persistent, bidirectional links between client and server instead of per-request HTTP connections. Each WebSocket connection counts as a single input, so functions not bound by CPU/GPU should allow concurrent inputs or Modal spins up a new container per connection. Modal auto-scales WebSocket handlers, targeting real-time streaming, progress updates on long-running tasks, and hosting frameworks like ComfyUI, Streamlit and Gradio.
Suno used Modal to scale inference and batch pre-processing across thousands of GPUs, bringing its music generation model to market four months early instead of hiring engineers to build and maintain infrastructure. The team ran batch pre-processing with a few short Python scripts and no config files, then expanded to model deployment with HTTP-exposed functions and chained inference across multiple models and containers. Modal's auto-scaling to thousands of GPUs handled holiday demand spikes without long-term GPU reservations.
Modal has added WebSocket support to its functions, enabling out-of-the-box hosting of frameworks like Streamlit, plus interactive commands via `modal container exec` for inspecting and debugging active containers. H100 GPUs are now available on Modal, and the client update adds `@build` lifecycle hooks, automatic background commits for Volumes, and expanded CLI commands including get, put and rm.
Modal has made NVIDIA H100 GPUs available to all users, priced at $7.65 per hour per GPU. NVIDIA benchmarks show the H100 delivers up to 4x training speedups and up to 30x inference speedups versus the A100, with 80 GB of on-chip DRAM and up to 8 GPUs linked via NVLink at 3.6 TB/s. Users enable it by setting "H100" as the GPU type in the Modal decorator.
Modal and Hugging Face's Text Embedding Inference server can embed the entire English Wikipedia in about 15 minutes for just over $15, using Modal's serverless GPU scaling. The post argues open-source embedding models beat API-based ones like OpenAI's text-embedding-ada-002 for production RAG, since they can be fine-tuned on user data and avoid rate limits and per-token costs.
Modal announced that its cloud function platform is now generally available, letting anyone sign up and run Python code in the cloud within minutes, after opening a waitlist in 2022. The platform offers serverless execution billed per second, autoscaling GPU inference endpoints on A100, A10G, T4, L4 and H100, and large-scale batch jobs across thousands of containers with no YAML configuration. Modal also raised a $16M Series A led by Redpoint Ventures, with Amplify Partners, which led the seed round, and plans to use the capital to accelerate its product roadmap.
Modal Labs has raised a $16 million Series A led by Redpoint Ventures, with existing investor Amplify Partners and new participants including Lux Capital and Definition Capital, bringing total funding to $23 million. The company also officially launched its cloud platform, previously in beta with early customers such as Ramp, Substack and SphinxBio, and will use the funds to accelerate product development and hire engineers at its New York City headquarters.
Cursor measured Llama-2-70B inference on 2 80-GB A100 GPUs and found it over 3x cheaper than GPT-3.5 for prompt tokens but far more expensive for generated tokens at small batch sizes. Prompt processing costs about $0.00042 per 1K tokens versus GPT-3.5's $0.0015, while generation reaches $0.066 per 1K tokens at batch size 1 and only approaches GPT-3.5 pricing at large batches that push time to first token to several seconds. Cursor recommends open-source models for prompt-heavy tasks like classification and reranking, and closed-source APIs for generation-heavy workloads.
Why it matters: Cursor's measured cost and latency data shows when self-hosting Llama-2-70B beats GPT-3.5 and when it does not, offering a reusable framework for open-source model serving decisions.
Cursor researcher Arvid Lunnemark argues that prompting should be called "prompt design" and compared to web design, since both pursue clear communication with dynamic inputs and must fit content to different "sizes" — screen sizes for the web, context windows for prompts. He released Priompt v0.1, a React-like JSX-based prompt design library already used inside Cursor, with a preview site for rendering prompts against real request data.