Skip to content
Only First-party
July 23, 2025Wednesday
  1. Modal Blog52

    Modal benchmarks open ASR models for batch transcription at 100x speed and cost gains

    Modal implemented a batch transcription service on its platform using NVIDIA's parakeet-tdt-0.6b-v2 and canary-1b-flash, reporting over 100x faster or 100x cheaper end-to-end performance than a popular proprietary ASR API while matching its error rate on roughly a week of ESB benchmark audio. The team attributes the gains to shuffling files across request batches, sorting audio by duration within GPU batches, parallel downloads, and empirically chosen GPU and worker configurations. The post walks through the architecture and optimization steps so others can build similar services.

July 16, 2025Wednesday
  1. Modal Blog30

    Dollars per token considered harmful: why self-hosted LLM inference should be priced per request

    A Modal Blog post argues that "dollars per token" pricing, inherited from API providers like OpenAI, Anthropic and Alphabet, is actively harmful for teams running self-hosted LLM inference. It contends that application users care about requests rather than tokens, and that per-request framing is needed to reason about latency, replica counts and compute costs.

July 11, 2025Friday
  1. Modal Blog40

    Modal Product Updates: Multi-node Training Clusters, B200 and H200 GPUs, and Client 1.0

    Modal has launched multi-node training clusters in beta, letting users tap dozens of GPUs across multiple hosts by adding the @clustered decorator, with 3.2 Tbps Infiniband RDMA interconnect for near-linear scaling. The platform now offers NVIDIA B200 and H200 GPUs serverlessly at $6.25/hr and $4.54/hr respectively, claiming up to 2-4× faster LLM inference than H100s. Modal also released Client 1.0, emphasizing API stability, alongside updates including a read-only Volume method and timezone support for Cron schedules.

  2. Modal Blog31

    Jamsocket is joining Modal

    Jamsocket, the backend platform behind the open-source orchestrator Plane, Y-Sweet and ForeverVM, is joining Modal, with co-founders Paul Butler and Taylor Baldwin joining the Modal team. Jamsocket products will keep operating normally while core functionality is migrated to Modal, which the founders say lets AI-centric customers deploy GPU inference, code sandboxes and real-time backends from one platform.

July 7, 2025Monday
  1. Modal Blog28

    How Modal Powered 250,000 Lovable App Creations in a Weekend

    Lovable used Modal Sandboxes to run LLM-generated code during a June 2025 promotional weekend with Anthropic, OpenAI, and Google, enabling users to build an estimated 250,000 applications in 48 hours. Modal ran over 1 million sandboxes with up to 20,000 concurrent at peak as Lovable's concurrent sessions surged 2.5-3x, and its platform team was not paged once. Lovable cut its sandbox orchestration code from 15,000 lines to 700 lines after switching to Modal.

July 2, 2025Wednesday
  1. Modal Blog40

    How Modal Used Evals and Inference-Time Compute Scaling to Generate Scannable QArt Codes

    Modal used evals and inference-time compute scaling to improve its QArt code system, boosting QR scan rate to its 95% service-level objective while improving aesthetic quality and returning codes in under 20s at p95. The work follows the standard ML playbook of collecting human judgments, aligning to them, then scaling, addressing the "it worked on my prompt" gap that leaves many generated QR codes unscannable.

June 30, 2025Monday
  1. Modal Blog28

    How Quora uses Modal to run thousands of Python sandboxes simultaneously

    Quora uses Modal Sandboxes to securely execute LLM-generated code in Poe, its AI chatbot platform, shipping the feature months earlier than an in-house build would have allowed. The team stress-tested Sandbox creation throughput to 1000 sandboxes per second and reports saving 2 engineers' worth of ongoing infrastructure maintenance time.

June 26, 2025Thursday
June 24, 2025Tuesday
  1. Vapi Blog22

    Mistral vs Llama 3: A Complete Comparison for Voice AI Applications

    Mistral and Llama 3 split on philosophy for voice AI: Mistral's 7B to Mistral Small 3.1 (24B) models favor low-latency efficiency, while Llama 3 scales from 8B to 70B for stronger reasoning and multilingual reach. Mistral Small 3.1 reaches 0.29s time to first token at 150-166 tokens per second, and Mistral Small 24B beats Llama 3.1 8B Instruct on ARC-C, GPQA and MMLU, while Llama 3.1 405B scores 85.2% on MMLU and 96.8% on GSM8K.

June 23, 2025Monday
  1. Vapi Blog22

    8 Alternatives to Azure for Voice AI STT on Vapi

    Vapi now offers eight speech-to-text providers — Gladia, AssemblyAI, Deepgram, Cartesia, Talkscriber, OpenAI, Speechmatics and ElevenLabs — as drop-in alternatives to Azure for voice agents. Switching providers takes a single JSON parameter change, letting developers benchmark latency, cost and compliance. Gladia claims sub-100 ms end-to-end latency, AssemblyAI bundles summaries and PII redaction in one API call, and Cartesia runs fully on-premises.

June 19, 2025Thursday
  1. Vapi Blog22

    Gemini Flash vs Pro: Comparing Google's Latest LLMs

    Google's Gemini 2.5 Flash and 2.5 Pro differ mainly in speed versus reasoning depth, both offering 1M-token context windows and native availability in Vapi for voice agents. Flash returns its first token in 0.21–0.37 seconds at 163 tokens per second and costs $0.15/$0.60 per million input/output tokens, while Pro costs $1.25/$10 and adds a Deep Think mode, with a 2M-token window planned.

June 18, 2025Wednesday
  1. Modal Blog58

    Modal shows how to run FLUX.1-dev three times faster

    Modal engineers describe making Black Forest Labs' FLUX.1-dev run about three times faster as an autoscaling service, cutting a 1024x1024 image from roughly 6.75 seconds to under three. Standard optimizations, namely the Torch compiler with max-autotune, fused QKV attention and channels-last memory layout, gave about 1.5x, while First Block Caching from the ParaAttention repo, based on the TEACache paper, added another 2x at a threshold of 0.12. Compile caches and Modal Memory Snapshots cut cold start latency by 30x.

  2. Vapi Blog22

    Claude vs ChatGPT: The Complete Comparison Guide

    Claude and ChatGPT are compared across conversational quality, context memory, speed, features, creative writing, research, pricing, and interface. Claude handles up to 200,000+ tokens for Pro users and excels at creative writing and nuanced conversation, while ChatGPT supports 128,000 tokens, responds in 2-3 seconds versus Claude's 5-10 seconds, and offers image generation and Custom GPTs. Both Pro tiers cost $20 per month, with Claude Max at $100-$200 and ChatGPT Pro at $200.

  3. Vapi Blog18

    Five Ways Real Estate Agents Can Use AI

    A guide outlines five ways real estate agents can use AI, including voice agents built on Vapi that answer calls, qualify leads, and book showings around the clock. It also covers automating content with ChatGPT and Canva, plus AI pricing tools such as HouseCanary and CoreLogic that produce valuations with confidence scores.

June 17, 2025Tuesday
  1. Vapi Blog38

    Deepgram Nova-3 vs Nova-2: How the STT Engine Evolved

    Deepgram's Nova-3 cuts streaming word error rate from Nova-2's 8.4% to 6.84% and reaches 5.26% batch WER, a 54.3% reduction in streaming errors. Nova-3 replaces Nova-2's separate domain-specific models with a single adaptable network and adds real-time multilingual conversation handling, up to 100 self-serve custom terms without retraining, and redaction for up to 50 entities. Both models run natively on Vapi with identical endpoints, so switching is a parameter change, though Nova-3 sits at Deepgram's premium pricing tier.

June 9, 2025Monday
  1. Modal Blog40

    Modal 1.0 Released: More Stable Python SDK with Explicit Configuration

    Modal has launched version 1.0 of its Python SDK client, a milestone the company says brings a new level of maturity and stability to its serverless cloud compute platform, with far fewer breaking changes expected going forward. The release removes "magic" behaviors such as automatic mounting of local packages, splits core functionality across separate decorators like the new @modal.concurrent, and consolidates local file handling under the modal.Image class. Modal also plans SDKs for other languages and is exploring how the platform should serve AI-driven agent workflows.

June 5, 2025Thursday
  1. Vapi Blog22

    GPT-4.1 vs Claude 3.7 Sonnet: Which AI Delivers Better Voice Agents?

    Vapi compares GPT-4.1 and Claude 3.7 Sonnet as the LLM behind voice agents, weighing conversation quality, context handling, reasoning transparency, platform integration and cost. GPT-4.1 offers a 1,000,000-token context window and 32,768-token outputs with 75% caching discounts, while Claude 3.7 provides a 200,000-token context and 128,000-token outputs. Both models are selectable on Vapi's voice agent platform.

June 4, 2025Wednesday
  2. Vapi Blog22

    Three Examples of Voice AI Improving Logistics

    Vapi outlines three voice AI use cases in logistics: ShipEX uses voice intelligence across 350 trucks to route maintenance alerts and accident reports, the Idaho State Liquor Division's voice picking system lifted pick rates 35%, and maintenance technicians log issues by voice to auto-create CMMS work orders. The article also details API integration with TMS, CRM, and WMS systems and deployment considerations.

  3. Vapi Blog22

    11 Great ElevenLabs Alternatives: Vapi-Native TTS Models

    Vapi now offers 11 built-in text-to-speech models as alternatives to ElevenLabs, including Neuphonic's Neu-hq and Neu-fast, Cartesia's Sonic lineup, Azure TTS, OpenAI TTS, Deepgram, Smallest AI's Lightning, LMNT, PlayHT, Hume's Octave and Rime AI's Mist v2. Developers can switch between them in minutes without re-coding, and Vapi says it handles over a million voice calls per day across these models.

May 30, 2025Friday
  1. Modal Blog42

    Modal adds Nvidia B200 and H200 GPUs at $6.25 and $4.54 per hour

    Modal now offers Nvidia B200 and H200 GPUs to any account holder with no sales contact or quota request, priced at $6.25/hour and $4.54/hour respectively. The B200 carries 180 GB of HBM3e memory and 8 TB/s bandwidth, and in Modal's vLLM test running DeepSeek V3 in 8-bit precision, 8xB200 cut median time-to-first-token 2.5x versus 8xH200 at 1 request per second.

  2. Vapi Blog15

    Best Speechify Alternatives: 5 TTS Tools Compared

    A comparison of five Speechify alternatives picks NaturalReader for cross-device use and app integrations at $119 per year, Voice Dream Reader for iPhone voice control at a $14.99 one-time price, and Murf AI for professional voices from about $19 per month. ReadSpeaker focuses on reading web pages across 45 languages from $4.90 per month, while Balabolka is a free Windows-only option.

May 29, 2025Thursday
  1. Vapi Blog25

    Choosing Between Gemini Models for Voice AI

    Vapi outlines how to pick among four Gemini models for voice agents: Gemini 1.0 Pro for predictable, compliance-friendly flows at $0.50 input and $2.00 output per million tokens, Gemini 1.5 Flash for high-volume work with a 1M token context window, Gemini 1.5 Pro for deep reasoning with a 2M token window and 99.7% long-context retrieval accuracy, and Gemini 2.0 Flash for native tool calling.

  2. Modal Blog22

    Twirl Joins Modal to Expand Data Orchestration and European Presence

    Modal announced that Twirl, a data orchestrator for developing, testing and deploying data pipelines, is joining the company. Twirl co-founders Rebecka Storm and Eric Hansander will join Modal's Stockholm office, where they will develop and expand Modal's serverless infrastructure and grow its European footprint. Modal said the team's data product experience will help it expand beyond AI inference into adjacent AI, ML and data use cases.

May 28, 2025Wednesday
May 26, 2025Monday
  1. Vapi Blog10

    Introduction to SRTP: Secure Real-Time Transport Protocol

    SRTP secures RTP voice and video streams by adding encrypted payloads, authentication tags, and a master key identifier to each media packet. It relies primarily on AES counter mode and HMAC-SHA1 authentication tags, with sequence numbers blocking replay attacks, and pairs with DTLS-SRTP or SDES for key exchange. SRTP is positioned as a lower-overhead, application-layer option for real-time communications, with GDPR, HIPAA, and CPRA cited as compliance drivers.

  2. Vapi Blog18

    Env Files and Environment Variables for Voice AI Projects

    A guide on using .env files and environment configuration in voice AI projects, covering dotenv setup for Node.js and Python, adding .env to .gitignore, and Vapi-specific keys like VAPI_API_KEY and VAPI_PROJECT_ID. It cites GitGuardian's finding of 12.8 million secrets leaked on GitHub in 2023, a 28% increase, and recommends separate env files per environment plus tools such as git-crypt, SOPS, AWS Secrets Manager and HashiCorp Vault.

May 23, 2025Friday
  1. Vapi Blog18

    Top 5 Character AI Alternatives for Seamless Voice Integration

    A Vapi blog roundup lists five Character AI alternatives for voice integration: Vapi, AnimaAI, JasperAI, Novel AI and AI Dungeon. Vapi is pitched as an API-native Voice AI platform supporting 100+ languages, connecting to 40+ applications, with Sesame-powered voice agents and automated test suites for catching hallucinations.

  2. Vapi Blog18

    Amazon Lex vs Dialogflow: Complete Platform Comparison Guide

    Amazon Lex suits teams already on AWS needing tight integration with Amazon services, while Dialogflow offers broader language support and an easier learning curve for newcomers. Lex uses Alexa's technology with pay-as-you-go pricing including a free tier of 10,000 text and 5,000 speech requests monthly for the first year, and Dialogflow supports over 30 languages with pre-built agents and sentiment analysis.

  3. Vapi Blog8

    Mastering Environment Variables for Vapi Voice AI Integration

    Vapi's guide explains how to configure environment variables for its Voice AI integration, covering temporary vs. permanent and user-level vs. system-level variables. It details setup on Windows, macOS and Linux with Vapi-specific examples like VAPI_API_KEY and VAPI_MULTILINGUAL_SUPPORT, plus naming and security best practices such as never hardcoding API keys.

  4. Vapi Blog18

    Narakeet: Turn Text Into Natural-Sounding Speech

    Narakeet converts written scripts into lifelike voiceovers, offering 600+ voices across 70+ languages and support for 140+ languages and dialects. The platform syncs narration with PowerPoint, Google Slides, and Markdown files, generates subtitles and captions, and provides a developer API with batch processing. Plans start around $5 per month, with a free tier available.

May 22, 2025Thursday
  1. Modal Blog58

    Modal launches Batch, processing 1 million jobs with one line of code

    Modal launched Modal Batch, a batch-processing interface backed by a rebuilt durable queue that can queue 1 million inputs at once, up from 2,000, and guarantees execution up to 7 days after a job is launched, up from 1 day. A new native handler, .spawn_map, spawns thousands of containers over a set of inputs without users managing async logic, and Harvey reports a 10x speed-up in its document-processing pipeline after fanning inputs across 1,000 containers. Suno uses it to run GPU-accelerated preprocessing on thousands of GPUs, and Achira relies on its retry mechanism for quantum mechanical datasets.

  2. Vapi Blog20

    The 10 Best Open-Source Medical Speech-to-Text Tools

    A guide ranks 10 open-source speech-to-text models for clinical use, evaluated on real-time accuracy, clinical terminology handling, operational reliability and data security. The list covers Vosk, DeepSpeech, SpeechBrain, OpenAI Whisper and ESPnet, matched to settings such as low-resource rural clinics, multilingual hospitals and terminology-heavy research centers. All can be integrated with Vapi's model-agnostic voice AI platform.

May 20, 2025Tuesday
  1. Modal Blog34

    Modal raises serverless KV store Dict limits to unlimited storage with new locking

    Modal upgraded its serverless Dict key-value store: new Dicts now have unlimited storage instead of the previous 10GiB cap, and item expiry changed to 7 days since last write or read. A new skip_if_exists flag on .put() enables distributed locking and "exactly once" semantics, and Dicts now offer data durability. The changes apply only to newly created Dicts.

  2. Vapi Blog22

    Vapi Integrates GoHighLevel Calendar Into Its Voice Agents

    Vapi has rolled out direct integration with GoHighLevel Calendars, embedding its voice agents into the GHL scheduling flow. Agents can now check real-time availability on connected GHL calendars, create contacts from caller details, and book appointments tied to the correct contact. Setup mirrors connecting Google Calendar in Vapi, with four new tools: Create Event, Check Availability, Create Contact, and Contact Get.

May 7, 2025Wednesday
April 30, 2025Wednesday
  1. Modal Blog38

    Modal launches alpha SDKs for JavaScript and Go

    Modal released alpha versions of its official SDKs for JavaScript and Go, letting developers call deployed Modal Functions and create isolated sandboxes from Node, Next.js, or Go services without writing Python. The SDKs cannot deploy functions, which still require Python, and only support JSON-like or bytes data types for now. Modal says it can run 10,000 sandboxes in either language without issues.

April 24, 2025Thursday
  1. Modal Blog22

    How Lemon Slice built real-time generative video with Modal and Daily

    Lemon Slice built Lemon Slice Live, a product for real-time video conversations with AI characters, using Modal and Daily's low-latency primitives to reach an end-to-end video response latency of 3-6 seconds. The pipeline runs a Pipecat server and a 1B parameter video model on separate Modal containers that communicate via Modal's Tunnel and are co-located with region selection, with Deepgram, Grok and ElevenLabs handling speech-to-text, LLM response and text-to-speech. Earlier, two Python Modal Functions let Lemon Slice scale to 10,000 requests per hour without managing an ECS cluster, job queue or load balancer.

April 22, 2025Tuesday
April 18, 2025Friday
  1. Modal Blog22

    How sync. uses Modal to lipsync 100 hours of video a day

    sync. is using Modal to lipsync over 100 hours of video a day, splitting long videos into scenes that run face detection and translation across n T4 GPU containers before passing each scene to its proprietary LipSync GenAI model on A100s. The team deploys up to 95 times a day and shipped 10 major model variants to production in one year, after moving off a Google Colab prototype and AWS Lambda.