Skip to content
Hot eventDeveloping

Bitdepth finds open models 95.7% cheaper on routine AI jobs

1 reports1 reporting sourcesUpdated 2 h ago

Start here

AI digest

Bitdepth ran five routine AI agent jobs on Claude Sonnet 5.5 and on the open models gpt-oss-120b and DeepSeek V4 Pro, and reported that a month of that work would cost $258.53 on Claude Sonnet 5.5 versus $11.04 using the cheapest setup that answered every job correctly — a 95.7% cut. The whole test itself billed $1.29. Four of the five jobs ran on the open models without a wrong answer. The fifth, an inbox digest, failed on both open models and was kept on Claude Sonnet 5.5. The jobs were made-up, single-call tasks with code-graded answers, and caching was left at each host's default. Bitdepth notes the sample is small and that real agents chain calls with longer history, so the gap may not carry over to production use.

Generated from reports · updated 2 h ago

Report timeline

Follow the reports to see the event from different sides.

November 20
  1. Hacker News · AI
    Bitdepth tests five routine agent jobs: $258.53 on Claude Sonnet 5.5 vs $11.04 on open models

    Bitdepth ran five routine agent jobs on Claude Sonnet 5.5 and on gpt-oss-120b and DeepSeek V4 Pro, finding a month of that work cost $258.53 on Claude Sonnet 5.5 versus $11.04 using the cheapest setup that got every answer right, a 95.7% cut. Four of the five jobs moved without a wrong answer, while the inbox digest failed on both open models and was kept on Claude Sonnet 5.5. The test used made-up, single-call jobs with code-graded answers, and the author notes the small sample and that real agents chain calls with longer history.

Heat for this event

Not enough continuous observations yet to draw a trend.