When Local LLMs Beat Claude's API for Claude Code - and When They Don't
- 9 min read
I’ve written three separate pieces on running Claude Code against a local model: the cost investigation, the setup guide, and the hardware benchmarks. None of them answer the actual question people ask, which isn’t “how” - it’s “should I.” Here’s the framework, pulled from those three pieces plus what’s changed since.
The number that decides this, and it isn’t tokens/sec
The single biggest factor isn’t model quality or hardware speed. It’s your prompt cache hit rate on Anthropic’s API.
Claude Code loads 30,000-60,000 tokens of context - CLAUDE.md, project instructions, codebase patterns - at the start of a session. Anthropic’s prompt caching stores that computed state server-side; every subsequent message in the session pays roughly 10% of normal input cost to hit that cache instead of re-sending everything. A 97.2% cache hit rate, which is realistic for a long working session on one project, means you’re effectively getting 10x the context capacity for what you’re paying.
Local inference has no equivalent. Not oMLX, not llama.cpp out of the box for cross-session persistence, not any open-source stack that matches what Anthropic offers server-side. Every message reprocesses the full context from scratch. Run the math and a session that costs $5 with cached cloud pricing can cost the compute-time equivalent of $40-50 running locally - before you count hardware cost at all.
This is why “I’m spending a lot on Claude Code, I should go local” is usually the wrong instinct on its own. The spend number doesn’t tell you your cache hit rate, and the cache hit rate is what the local-vs-cloud math actually turns on.
When local wins anyway
Very heavy, sustained, parallel usage. If you’re running multiple long-lived sessions across several projects daily - not occasional bursts - the hardware cost amortizes fast enough to beat even well-cached cloud spend. The working hypothesis from the cost investigation: for that usage pattern, a high-memory Mac Studio can pay for itself in 2-4 weeks and cut ongoing costs 85-90%. That’s a hypothesis, not a settled result - it depends on how well the remaining gap (screenshot review, subtle debugging, recovering from a wrong path) gets closed by tooling, which is still an open question.
Data that can’t leave your network. If you’re working with regulated data - PII, health records, anything under GDPR/HIPAA/SOC 2 - the conversation isn’t about cost at all. It’s whether cloud AI tools are on the table. That’s a governance decision, not a token-math one.
You already own the hardware. If a capable machine is sitting idle - a 24GB+ GPU, a 128GB Mac Studio or Strix Halo box someone already bought - the marginal cost of using it for Claude Code is close to zero, and the cache-hit math that favors the cloud API stops mattering as much.
When cloud wins, which is most of the time
Anything below very heavy, sustained usage. Occasional or moderate Claude Code use, especially on a small number of projects, almost never clears the bar. The cache advantage compounds every session; hardware cost doesn’t shrink because you used it less.
Quality-sensitive work. Purpose-built local coding models have closed a lot of the gap on straightforward implementation, but the remaining difference shows up exactly where you’d expect: subtle debugging, recovering from a wrong path mid-task, and anything requiring visual/screenshot review. The gap is still real on SWE-bench Verified: Anthropic’s top model sits at 0.950, and the best-scoring open-weight model you could actually self-host, DeepSeek-V4-Pro-Max, sits at 0.806 - good enough for most implementation work, not a wash. If quality-sensitive work is a meaningful slice of your workload, local models cost you more in review time than they save in compute.
You haven’t measured your actual cache hit rate. If you don’t know this number, get it before deciding anything else - tools like CodeBurn can break down spend by project and cache performance. A low cache hit rate changes this whole calculation; a workflow that’s already inefficient on the cloud side won’t automatically get cheaper by moving local, it’ll just move the inefficiency somewhere with a bigger upfront cost.
The actual decision
Measure your cache hit rate and your usage pattern first. If you’re at very heavy, sustained, multi-project usage and a high cache hit rate isn’t already making the cloud side cheap, that’s the profile where local pencils out - and at that point, the hardware benchmarks and setup guide tell you what to buy and how to configure it. Anything short of that profile, the cache math is doing more work for you on the cloud side than a local box will.
Related Reading
- From $500/Day to $0: Investigating Local LLM Inference for Heavy Claude Code Users
- Claude Code Local LLM Speed by Hardware Tier
- Running Claude Code with a Local LLM in 2026: No Proxy Required
Trying to figure out if local inference actually pencils out for your team? Let's talk through the tradeoffs.
Schedule a Call