Dollars per token considered harmful: why self-hosted LLM inference should be priced per request

A Modal Blog post argues that "dollars per token" pricing, inherited from API providers like OpenAI, Anthropic and Alphabet, is actively harmful for teams running self-hosted LLM inference. It contends that application users care about requests rather than tokens, and that per-request framing is needed to reason about latency, replica counts and compute costs.