llama.cpp v1.0 benchmarked on RTX 3090: gains only in parallel and speculative decoding
Start here
Homebrew Server Club benchmarked llama.cpp v1.0.0 (build 12359) against a pre-v1 build (12199) on an RTX 3090 and found single-request speed unchanged within 2%. Four parallel requests produced 14% more text per second with the 9B model and 22% more with the 27B model, and speculative decoding with MTP heads made the 27B model 52–55% faster. The gains require an NVIDIA RTX 30-series (Ampere) or newer GPU, so owners of older NVIDIA cards, AMD cards or Macs should not expect them. The test notes that a single user chatting without speculative decoding sees no speedup, so updating can wait. v1.0 removes the --defrag-thold flag, which must be dropped from commands, and requires an API key for tools or MCP servers on non-loopback addresses. Ollama 0.40.16 still ships an older llama.cpp build.
Generated from reports · updated 1 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIllama.cpp v1.0 benchmarked on an RTX 3090: up to 55% faster with speculative decoding
A hands-on test of llama.cpp v1.0.0 (build 12359) against a pre-v1 build on an RTX 3090 found single-request speed unchanged within 2%, while four parallel requests produced 14% more text per second with the 9B model and 22% more with the 27B model. Speculative decoding with MTP heads made the 27B model 52 to 55% faster, and the gains require an NVIDIA RTX 30-series card or newer. The update removes the --defrag-thold flag and requires an API key for tools or MCP servers on non-loopback addresses, and Ollama 0.40.16 still ships an older llama.cpp build.
Heat for this event
Not enough continuous observations yet to draw a trend.