Modal, Pipecat and open models achieve one-second voice-to-voice latency

Modal's Ben Shababo walks through building a conversational voice bot on Modal with the open source Pipecat framework and open weights models, reaching a median voice-to-voice latency of about one second when the client and containers are geographically close. The pipeline uses NVIDIA parakeet-tdt-0.6b-v3 for STT, Qwen3-4B-Instruct-2507 served with vLLM, KokoroTTS for speech output, and ChromaDB with all-minilm-l6-v2 for RAG over Modal's docs. Modal Tunnels bypass the input plane for low-latency WebSocket and HTTP links to the STT, TTS and vLLM services, while a FunctionCall tied to each conversation session recovers autoscaling, and pinning services to nearby regions cuts network latency.