Skip to content
Baseten Blog·· 1 h agoPicksAI score71

Baseten brings deferred tool loading to open models, cutting coding agent cost by 62.5%

Deferred tool loading: Make your agent 62.5% cheaper without changing your model or harness

AI summary

Baseten added deferred tool loading to open models (DeepSeek, Kimi, GLM) on Baseten Model APIs, supporting both Chat Completions and Messages API formats. In Claude Code with 3 MCPs enabled, coding benchmarks ran 62.5% cheaper and 26.2% faster on average, while agentic benchmarks were 51.1% cheaper and 14.9% faster. Instead of injecting every tool schema into the system prompt, the harness passes tool names plus a tool_search tool (~1.3k tokens versus ~141k tokens for 3 MCP servers) and renders the full schema inline only when needed, keeping the KV cache prefix valid.

Why it matters

Baseten adds deferred tool loading for open models, with benchmark numbers showing how much of an agent's cost comes from preloaded tool schemas.

Source:Baseten Blog · baseten.co