Skip to content
Only First-party#Open-source ecosystem
September 16Wednesday
  1. Mistral AI55

    Mistral partners with Mozilla to power Firefox Smart Window in France and North America

    Mistral announced a partnership with Mozilla under which its models now power Firefox Smart Window (beta), Mozilla's AI browsing assistant, for users in France and North America, with the UK and Germany expected later this year. Mistral says Smart Window conversations are not saved on Mozilla's servers by default and that partners agree to zero data retention, and it frames the deal as bringing open-weight, regionally fine-tuned models to consumer browsing.

September 14Monday
  1. Modal Blog40

    Modal Product Updates: Sandbox Sidecars, Kimi K3 and GLM 5.3 Support, Refreshed Dashboard

    Modal added day-zero support for four open-weight models — Kimi K3, Qwen3.8-2.4T-A95B, GLM-5.3, and GLM-5.3-Flash — on its Auto Endpoints, with Kimi K3 running at 460 tokens per second on agentic workloads via a custom DFlash speculator. Sandbox Sidecars entered public alpha, VM Sandboxes reached public beta, and the broad-region pricing multiplier dropped from 1.5x to 1.15x. The dashboard was refreshed with a light mode beta, and Python SDK 1.5.5 plus JavaScript and Go SDK 0.10.0 shipped.

August 18Tuesday
  1. LiveKit Blog40

    LiveKit: multi-participant robotics beyond 1:1 connections

    LiveKit is extending its SFU architecture beyond single peer-to-peer links so robot deployments can run multi-participant sessions, where a Room lets each camera, model, operator, or service join as its own participant. LiveKit describes three architectures: human-in-the-loop data recording with robot, policy, operator, and recorder in one session; VLM-guided skill orchestration where a vision-language model selects skills at roughly 1 Hz while separate policy participants control the robot at tens of hertz; and policy escalation, where a reward model at 30 Hz or higher flags stuck or unsafe states and the robot revokes authority from the specialist policy.

August 13Thursday
  1. MiniMax Blog71

    MiniMax releases Music 3.0, an open-weights music generation model for full songs up to five minutes

    MiniMax introduced Music 3.0, an open-weights music generation model that composes, arranges and produces a complete song of up to five minutes from a creative concept and optional lyrics in a single generation. The pipeline pairs an 8B Global LLM initialized from Qwen3.5-8B with a 0.6B Local LLM, fuses their continuous hidden states to condition a 2.4B flow-matching module and a 123M Flow-VAE, and uses eight-layer RVQ plus Structured Captions that describe emotion, instrumentation and vocal delivery over time. MiniMax says this yields more accurate interpretation of creative intent, fuller arrangements and clearer, more natural vocals.

    Why it matters: MiniMax's architecture breakdown shows how an 8B global LLM paired with a 0.6B local model and flow-matching stack targets the long-song coherence problem.

August 4Tuesday
  1. Modal Blog34

    Modal moves serverless Functions to a regional I/O plane, cutting p50 latency by ~80ms

    Modal has migrated all Functions to a new geographically distributed I/O plane, dropping end-to-end Function Call latency by roughly 80ms at p50. Previously every Function's inputs and outputs routed through us-east I/O servers, adding up to a few 100ms per call; the new plane is available in ≥4 regions and selectable via the routing_region flag. The server layer was rewritten in Go, with non-critical work moved off the hot path.

July 27Monday
  1. Modal Blog75

    Moonshot's Kimi K3 now available on Modal with day-zero vLLM support

    Modal made Moonshot's Kimi K3 available on its platform on release day, offering the 2.8 trillion parameter multimodal model with a 1M token context window through a token-priced Shared API and dedicated Auto Endpoints. Modal says a custom DFlash speculator tuned to K3's architecture raises interactivity from 100 to 460 tokens per second and throughput from 800k to 1.5 million TPM per GPU on agentic workloads. The model uses a mixture-of-experts design with 16 of 896 experts active per token, and Moonshot contributed a new prefix-caching implementation to vLLM ahead of the release.

    Why it matters: Modal details how it brought Moonshot's 2.8T-parameter Kimi K3 to its platform on release day, including a custom DFlash speculator that lifts interactivity from 100 to 460 tokens per second.

June 23Tuesday
  1. Modal Blog60

    Modal launches Auto Endpoints for self-serve, OpenAI API-compatible LLM inference

    Modal launched Auto Endpoints, a self-serve service that deploys production-grade LLM inference as an OpenAI API-compatible endpoint backed by a Modal App the user can see and control. Each supported model ships with a starting deployment tuned from Modal's own benchmarking, using engines like SGLang and the DFlash speculative decoding drafter from Z Lab, and users can inspect GPU choices, engine flags and inference metrics such as TTFT, ITL and speculative decoding acceptance length. Modal also released Modal Servers from beta for regionalized routing with 5ms overhead, and said it generates endpoint deployments with an internal agentic system still monitored by human engineers.

    Why it matters: Modal's Auto Endpoints expose the serving stack, engine metrics and benchmark recipes that managed inference providers keep hidden, giving teams a self-serve path to owning open-model inference.

June 19Friday
  1. Modal Blog60

    Modal releases DFlash speculators for Qwen 3.5 and 3.6, claiming 5-20% extra speedup

    Modal released a set of DFlash draft models for the Qwen series on Hugging Face, including Qwen 3.6 35B-A3B-DFlash and Qwen 3.5 variants from 4B to 122B-A10B, plus a DFlash speculator for Qwen 3.5 397B-A17B built with SGLang. The company says the new drafters add 5-20% speedup over existing DFlash baselines and can run Qwen 3.5 122B-A10B at over 1000 tok/s at concurrency 1 on a B200 node, versus 250 tok/s without speculation. The post argues speculative decoding delivers 2x-3x gains where kernel optimization yields small percentage points, and presents a roofline model for estimating speedup from acceptance length.

    Why it matters: Modal explains why it bets on speculative decoding and releases DFlash draft models for the Qwen series, with measured speedups and a roofline model for estimating gains.

June 11Thursday
  1. Modal Blog34

    Modal Makes FlashAttention-4 Faster for LLM Inference

    Modal contributed optimizations to the FlashAttention-4 kernel to speed up decode-heavy LLM inference workloads, where memory bandwidth is the bottleneck. The changes include porting the "split KV" parallelism technique to FA4 and replacing TMA cp.async.bulk loads with cp.async for irregular memory accesses. An FP8 input PR (merged April 17, 2026) delivers up to 1.16x throughput over the bf16 baseline and enables smaller KV caches.

June 1Monday
  1. MiniMax Blog87

    MiniMax M3 released with 1M context, frontier coding and native multimodality

    MiniMax released M3, an open-weight model it says is the first to combine frontier coding and agentic performance, a 1M-token context window, and native image and video input. M3 uses MiniMax's new MSA sparse attention architecture, which the company reports cuts per-token compute at 1M context to 1/20 of the previous generation and speeds prefill by more than 9x and decoding by more than 15x. It scores 59.0% on SWE-Bench Pro, 66.0% on Terminal-Bench 2.1 and 74.2% on MCP Atlas, and is available through MiniMax Code, the Token Plan and the API, with weights and a technical report promised within 10 days.

    Why it matters: MiniMax M3 combines a 1M-token context, frontier coding and agentic scores, and native image and video input in one open-weight model, with MSA cutting per-token compute at long context.

May 4Monday
February 11Wednesday
  1. Modal Blog78

    Modal 上线 GLM-5 免费端点并发布自部署与前端接入指南

    Modal 推出 GLM-5 的免费 OpenAI API 兼容端点，可用至 4 月底，限单个并发请求，同时发布在单节点 8 张 B200 上用 SGLang 部署的可复现代码。

    Why it matters: Modal 为 Z.ai 的 GLM-5 提供免费 OpenAI 兼容端点，并给出在 OpenCode、OpenClaw、Claude Code 和 Vercel AI SDK 中接入的具体配置。

December 2, 2025Tuesday
  1. Modal Blog42

    Modal adds Day 0 support for Mistral 3 with GPU snapshotting for 10x faster cold starts

    Modal now offers Day 0 support for Mistral 3, letting developers deploy the open model family on serverless GPUs without managing infrastructure. On the 3B Ministral 3 model, Modal's GPU snapshotting feature cut median cold start time from about 118s to about 12s, nearly a 10x reduction. Ministral 3 comes in 3B, 8B, and 14B sizes and performs competitively with the Qwen 3-VL series on benchmarks.

November 18, 2025Tuesday
  1. Modal Blog36

    Modal Blog: Host Overhead Is Killing Your Inference Efficiency

    Modal's blog explains how host overhead—gaps where the GPU sits idle waiting for the CPU to prepare work—shows up as low GPU kernel utilization in AI inference. At 50% host overhead, GPU inference costs are 2x what they could be. The post details fixes used in production open source inference engines, including constructing tensors on-device instead of transferring them and reducing unnecessary CUDA kernel launches.

November 4, 2025Tuesday
  1. Modal Blog58

    Modal, Pipecat and open models achieve one-second voice-to-voice latency

    Modal's Ben Shababo walks through building a conversational voice bot on Modal with the open source Pipecat framework and open weights models, reaching a median voice-to-voice latency of about one second when the client and containers are geographically close. The pipeline uses NVIDIA parakeet-tdt-0.6b-v3 for STT, Qwen3-4B-Instruct-2507 served with vLLM, KokoroTTS for speech output, and ChromaDB with all-minilm-l6-v2 for RAG over Modal's docs. Modal Tunnels bypass the input plane for low-latency WebSocket and HTTP links to the STT, TTS and vLLM services, while a FunctionCall tied to each conversation session recovers autoscaling, and pinning services to nearby regions cuts network latency.

October 30, 2025Thursday
  1. Modal Blog38

    Modal and Datalab: Deploy Marker and Surya Document Intelligence in Under 5 Minutes

    Modal has partnered with Datalab to let developers deploy the Marker document-parsing model and Surya OCR toolkit on Modal GPUs in under five minutes. Marker is a sub-billion-parameter model supporting over 90 languages, and running it on Modal reaches about 2.2 pages per second per container, roughly 10x the 0.22 pages per second on an M4 Mac using Apple MPS. Datalab's tools stay free for research, personal use, and startups under $2M in funding or revenue, with commercial licensing available.

July 23, 2025Wednesday
  1. Modal Blog52

    Modal benchmarks open ASR models for batch transcription at 100x speed and cost gains

    Modal implemented a batch transcription service on its platform using NVIDIA's parakeet-tdt-0.6b-v2 and canary-1b-flash, reporting over 100x faster or 100x cheaper end-to-end performance than a popular proprietary ASR API while matching its error rate on roughly a week of ESB benchmark audio. The team attributes the gains to shuffling files across request batches, sorting audio by duration within GPU batches, parallel downloads, and empirically chosen GPU and worker configurations. The post walks through the architecture and optimization steps so others can build similar services.

July 11, 2025Friday
  1. Modal Blog31

    Jamsocket is joining Modal

    Jamsocket, the backend platform behind the open-source orchestrator Plane, Y-Sweet and ForeverVM, is joining Modal, with co-founders Paul Butler and Taylor Baldwin joining the Modal team. Jamsocket products will keep operating normally while core functionality is migrated to Modal, which the founders say lets AI-centric customers deploy GPU inference, code sandboxes and real-time backends from one platform.

June 9, 2025Monday
  1. Modal Blog40

    Modal 1.0 Released: More Stable Python SDK with Explicit Configuration

    Modal has launched version 1.0 of its Python SDK client, a milestone the company says brings a new level of maturity and stability to its serverless cloud compute platform, with far fewer breaking changes expected going forward. The release removes "magic" behaviors such as automatic mounting of local packages, splits core functionality across separate decorators like the new @modal.concurrent, and consolidates local file handling under the modal.Image class. Modal also plans SDKs for other languages and is exploring how the platform should serve AI-driven agent workflows.

May 22, 2025Thursday
  1. Vapi Blog20

    The 10 Best Open-Source Medical Speech-to-Text Tools

    A guide ranks 10 open-source speech-to-text models for clinical use, evaluated on real-time accuracy, clinical terminology handling, operational reliability and data security. The list covers Vosk, DeepSpeech, SpeechBrain, OpenAI Whisper and ESPnet, matched to settings such as low-resource rural clinics, multilingual hospitals and terminology-heavy research centers. All can be integrated with Vapi's model-agnostic voice AI platform.

May 20, 2025Tuesday
  1. Modal Blog34

    Modal raises serverless KV store Dict limits to unlimited storage with new locking

    Modal upgraded its serverless Dict key-value store: new Dicts now have unlimited storage instead of the previous 10GiB cap, and item expiry changed to 7 days since last write or read. A new skip_if_exists flag on .put() enables distributed locking and "exactly once" semantics, and Dicts now offer data durability. The changes apply only to newly created Dicts.

December 28, 2024Saturday
  1. Modal Blog22

    Modal product updates: faster memory snapshotting, OIDC, async job queues

    Modal shipped a product update round led by a major upgrade to its beta memory snapshotting feature, which it says can cut cold start times by half for models like Stable Diffusion. The platform also added OpenID Connect support for short-lived identity tokens, eStargz image compression for faster builds from registries like ECR and Docker Hub, and is expanding async job queues toward a 1 million input limit, up from 2k.

November 24, 2024Sunday
  1. Modal Blog22

    Modal Signs Strategic Collaboration Agreement with AWS for GPU-Accelerated Generative AI

    Modal signed a Strategic Collaboration Agreement with AWS to help businesses develop and deploy AI products faster on its serverless platform for GPU-accelerated workloads, which can spin up GPU-enabled containers in as little as one second. As part of the deal, Modal will list on AWS Marketplace and invest in integrations such as AWS PrivateLink to serve customers with specific security and privacy requirements.

March 14, 2024Thursday
  1. Modal Blog36

    Modal's modal-http: Serving Web Functions on Serverless Infrastructure

    Modal built modal-http, an HTTP and WebSocket stack that lets its serverless functions accept web requests, with full WebSocket support recently completed. Unlike AWS Lambda's 15-minute and 50 MB limits, Modal containers can use up to 64 CPUs, 336 GB of memory and 8 Nvidia H100 GPUs, so the team translates HTTP into function calls rather than fronting containers with a conventional reverse proxy.

February 27, 2024Tuesday
  1. Modal Blog38

    Modal adds WebSocket support for real-time apps

    Modal now supports WebSocket connections, letting functions on its platform maintain persistent, bidirectional links between client and server instead of per-request HTTP connections. Each WebSocket connection counts as a single input, so functions not bound by CPU/GPU should allow concurrent inputs or Modal spins up a new container per connection. Modal auto-scales WebSocket handlers, targeting real-time streaming, progress updates on long-running tasks, and hosting frameworks like ComfyUI, Streamlit and Gradio.