Modal and CMU's Full Stack Data Lab release Quail, an AI-SQL inference engine that jointly optimizes query planner and inference

Modal and Carnegie Mellon University's Full Stack Data Lab built Quail, a query-aware inference engine for AI-SQL workloads that jointly optimizes the SQL query planner and the inference engine. On one multi-join query it processes over a billion tokens per minute per H100 GPU, more than 10x faster than Modal's vLLM baseline on the same hardware, at under 6¢ per billion tokens; on the team's new AI-SQL benchmark it runs 1.84x faster than vLLM geometrically averaged over tasks. Because AI-SQL filters and joins need only a single token per sequence, Quail skips decode, sampling and prefill-decode disaggregation, and it uses the known query plan to order requests and manage KV cache eviction.