Claude Code Local LLM Speed by Hardware Tier: What Backends Actually Claim

Every local-LLM setup guide, including my own, tells you which model to pull for your hardware. Almost none of them tell you what speed to actually expect once Claude Code’s agentic loop is hammering the backend with tool calls, not a single chat prompt. I haven’t run a controlled benchmark rig across every tier myself, so what follows is each backend’s own published throughput claims, organized by hardware tier, with the caveats that matter for agentic use specifically. Treat these as a starting expectation to validate on your own hardware, not a guarantee.

16 GB Unified Memory (M-series MacBook, base tier)

At this tier, the ceiling isn’t the backend, it’s the model. Ollama runs qwen2.5-coder:14b and gemma3:12b comfortably; oMLX’s own docs claim its tiered cache lets a 16 GB machine run models that would otherwise need closer to 24 GB, at some cost to speed on cache misses. Anything meaningfully above ~14B parameters starts swapping to oMLX’s SSD tier or fails outright on plain Ollama.

24 GB Unified Memory / VRAM (mid-tier)

Ollama’s own numbers put qwen3-coder (30B, 3B active) as a comfortable fit at this tier. Rapid-MLX’s own README claims up to 3x aggregate decode throughput over Ollama on Apple Silicon at concurrency, with some secondary coverage citing up to 4.2x, figures from their own benchmarks, not independently verified here. Its 27 tool-call parser modules are the more relevant advantage for Claude Code specifically: dedicated parsing across a session of repeated tool calls matters more than raw tokens/sec on a single prompt.

NVIDIA GPU, 16 GB VRAM (ExLlamaV3 + TabbyAPI)

ExLlamaV3’s EXL3 quantization format is documented as running Llama-3.1-70B at 1.6 bits-per-weight in under 16 GB VRAM, a real capability, per the project’s own benchmarks. In practice, a well-quantized 32B model (Qwen2.5-Coder-32B, for one) is likely to round-trip through Claude Code’s loop faster than a maximally-quantized 70B one, since at 1.6 bpw you’re trading model size for speed, not getting both.

128 GB Unified Memory (Strix Halo, Mac Studio M5 Max)

Two more options land in this tier, with a Linux/Windows split worth knowing before you buy.

AMD’s Ryzen AI Max+ 395 (“Strix Halo”) ships in 64 GB and 128 GB configurations. The 64 GB tier’s factory-default GPU carve-out is 32 GB, comfortable for Llama 3.3 70B at Q4_K_M per community benchmarks, but tight for 120B-class models or long context. On Linux, the 128 GB tier can address roughly 108 GB of that pool through a shared GTT allocation; on Windows, the GPU is limited to a fixed BIOS carve-out with no equivalent, so the large-model case is effectively Linux-only today. Pricing runs from around $1,499 for a 64 GB EVO-X2 up to $2,199-$3,649 for 128 GB builds, depending on vendor.

Apple’s M5 Max, in the August 2026 Mac Studio refresh, tops out at 128 GB unified memory with up to 614 GB/s bandwidth on the higher-core-count configuration, up from 546 GB/s on M4 Max. That bandwidth gain matters more than the headline TFLOPs for agentic use, since Claude Code’s tool-call loop is dominated by repeated context re-reads, not single large-batch inference.

Neither option changes the guidance in the tiers above, they extend the ceiling for people who need 70B+ models or very long context, they don’t make an underpowered tier faster.

Buying Around the 2026 RAM and GPU Shortage

Two supply-side stories from this year change the math if you’re buying hardware for this rather than using a machine you already own:

DRAM prices roughly doubled through 2026 on AI-datacenter demand, which raises the effective cost of any build that leans on system RAM, including Strix Halo’s shared pool. A used RTX 3090, with 24 GB of dedicated GDDR6X, enough for a 30B model at 4-bit, sidesteps most of that, since it isn’t competing in the same DRAM market. If you already own one, it’s a better value-per-GB right now than it looks on paper.

The RTX 50 Super refresh’s 24 GB tier has slipped repeatedly through 2026, with the current rumored window pushed toward late 2026 or CES 2027, driven by GDDR7 supply and cost pressure after Micron wound down 2 GB GDDR7 module production. If you were waiting for a 24 GB Super card before buying into the NVIDIA 16 GB VRAM tier above, budget for that wait or buy a used 3090 now and reassess later.

New in llama.cpp: Streaming MoE Experts from Disk

Worth flagging if you’re running a MoE model (Qwen3-Coder, DeepSeek-class) on a memory-constrained box: llama.cpp’s --n-cpu-moe flag, plus an in-progress expert-streaming feature, let the router prefetch only the experts a given token needs straight from disk, overlapping that I/O with compute on the previous layer, a claimed 2x+ throughput improvement over naive offload in the project’s own discussion threads. It isn’t stable everywhere yet, so treat it as something to watch on a memory-constrained NVIDIA or Strix Halo box, not something to plan a purchase around today.

What Actually Slows Down the Agentic Loop

A backend’s own benchmark number rarely matches what you see inside Claude Code, for three reasons:

Tool-call parsing overhead. Every tool call round-trips through a parser before the model sees the result. Backends that ship dedicated tool-call formats (Rapid-MLX’s 27 parser modules, for one) cut this overhead measurably versus a generic OpenAI-compatible bridge like LiteLLM.

Context re-processing. Claude Code re-sends accumulated context on every turn. Below 64K context support, you’ll see the model re-truncating and losing earlier instructions, which shows up as repeated or drifting behavior, not just slower output.

Cache misses on long sessions. oMLX’s two-tier cache is the exception, but on Ollama and Rapid-MLX, a long session with a lot of file reads will start evicting earlier context, and every cache miss costs a full re-read at un-cached speed.

The Practical Takeaway

If you’re on 16 GB unified memory, don’t fight the tier, Gemma 3 12B or Qwen2.5-Coder:14b on Ollama is the realistic starting point. At 24 GB, Rapid-MLX’s claimed throughput advantage is worth testing against Ollama on your own hardware before you commit to it. On NVIDIA hardware, a well-quantized 32B model is the safer default over a maximally-quantized 70B one for anything Claude Code is doing turn-by-turn. If you’re shopping for 70B+ headroom, a 128 GB Strix Halo box on Linux or an M5 Max Mac Studio are the current options, and a used RTX 3090 is worth a look before either, given where DRAM and GDDR7 pricing sit right now.

None of this changes the guidance in the 2026 setup guide on which backend to install. It just tells you what to actually validate before you assume a benchmark claim holds for agentic, tool-call-heavy use.

---

Related Reading

• Running Claude Code with a Local LLM in 2026: No Proxy Required