Qwen3:14b Fits in 12GB — So Why Does It Keep Getting Slower? | Intel Arc B580 Local LLM Part 4
When I measured the Intel Arc B580 across model sizes in Part 3, one result refused to make sense. A 14B model whose weights fit inside 12GB ran more than three times slower than a larger 12B model.
This article is me chasing that down. The answer first: the cause was not the size of the weights but the context length — how much input the model can hold at once. Set the context long enough and its working memory eats the VRAM; whatever does not fit spills onto the CPU, and the speed collapses. How easily that happens turned out to vary enormously between models.
Everything here was measured on the B580 alone, seated directly in a desktop PCIe slot (the RTX 3090 in the same machine took no part in this experiment), using Ollama on Vulkan. As of July 2026.
The previous article is here
https://kimono-oyaji.com/arc-b580-size-speed-power-en/The machine
I added the Intel Arc B580 to the desktop below.
| CPU | Ryzen 9 3950X |
| Motherboard | X570 |
| Memory | 64GB |
| GPU | NVIDIA GeForce RTX 3090, Intel Arc B580 (added for this test) |
| OS | Ubuntu |
The puzzle: it fits, so why is it slow?
In Part 3, qwen3:14b (5.9GB of weights in VRAM) managed only 9.14 tok/s of decode — the speed at which text is generated. In the same table, gemma4-12b (6.9GB) did 29.95 tok/s. The smaller model, the one that should have fit more comfortably, came in at under a third of the bigger one. The ordering was upside down.
My placement check at the time could only confirm that the work was not on the RTX 3090. It could not see a spill to the CPU. So I measured again, this time in a way that makes the spill directly visible.
The test: vary the context length, watch the split
I changed Ollama’s num_ctx (the context length it can hold at once) across four steps from 2,048 to 32,768, and at each step recorded:
- decode (tok/s) — how fast text is generated
- the GPU / CPU split that Ollama reports — that is, how much fell to the CPU
To make sure any overflow could only go to the CPU, I showed Ollama the B580 and nothing else, hiding the RTX 3090 completely (excluding it from both Vulkan and CUDA through environment variables). Throughout the run the 3090’s VRAM usage stayed at 285MB from start to finish, exactly as intended: unused.
As a control I ran gemma4-12b, which behaved normally in Part 3, under the same conditions.
Result: qwen3:14b slid onto the CPU as the context grew
| Model | Context | decode | GPU / CPU split |
|---|---|---|---|
| qwen3:14b (5.9GB of weights) |
2,048 | 36.11 | 100% GPU |
| 8,192 | 28.64 | 93% GPU / 7% CPU | |
| 16,384 | 13.83 | 82% GPU / 18% CPU | |
| 32,768 | 9.00 | 67% GPU / 33% CPU | |
| gemma4-12b (6.9GB of weights, control) |
2,048 | 30.54 | 100% GPU |
| 8,192 | 30.51 | 100% GPU | |
| 16,384 | 30.50 | 100% GPU | |
| 32,768 | 30.57 | 100% GPU |
qwen3:14b decode speed (the longer the context, the slower it gets, tok/s)
A longer context makes the KV cache squeeze VRAM until it spills to the CPU. gemma4-12b stayed 100% on GPU at a steady 30.5 tok/s across every context length.
The split is unmistakable. As the context grows, qwen3:14b hands more and more to the CPU, sliding from 36 down to 9 tok/s. The 9.00 tok/s at a context of 32,768 lines up almost exactly with the 9.14 tok/s from Part 3. That slowness was never about the model being large. Its default context length was large, and the excess was spilling onto the CPU.
[Added 2026-07-31] This test only went up to a context of 32,768. I extended it afterwards, and past 32,768 the decline stops — it levels off. On the B580: 7.86 tok/s at 65,536 and 8.03 tok/s at 131,072. Another GPU (an RTX 3060 over Thunderbolt) showed the same shape: 20.09 at 32,768, 17.87 at 65,536, 18.07 at 131,072. The share going to the CPU saturates, so it does not keep getting slower without limit.
Worth noting the other direction too: with the context set short (2,048) the model sat entirely on the GPU and hit 36 tok/s, beating gemma4-12b’s 30.5. A 14B model outrunning a 12B one is perfectly ordinary — as long as it fits.
The control, gemma4-12b, stayed 100% on the GPU at a flat 30.5 tok/s even at a context of 32,768. On the same 12GB card, how readily a model overflows depends entirely on the model.
Why 12GB is not a question of weight size alone
The weights are not the only thing occupying VRAM. There is also the working memory that grows as the model reads further — the KV cache, the area that holds on to what has been read so far — and it grows with the context length.
The gap between qwen3:14b and gemma4-12b comes down to how thick that working memory is for each model. The gemma4 family is designed to keep it small (partly through mechanisms that only look at a nearby window), so even a long context does not overflow. qwen3:14b carries a much thicker one, and stretching the context takes it past 12GB early.
I also tried compressing that working memory to see whether it would solve the problem. The result, up front: the setting did nothing on Ollama 0.30.x. Switching OLLAMA_KV_CACHE_TYPE between f16, q8_0 and q4_0 produced speeds that matched exactly, and the internal logs showed the setting was never passed through to llama-server. It is not that compression fails to help — the setting itself is ignored. To actually try it you have to drive llama.cpp directly and pass --cache-type-k / --cache-type-v.
There are also two distinct kinds of overflow. One is the kind here, overflowing on context. The other is overflowing on weights — the model simply does not fit in 12GB to begin with. The nemotron-3-nano:30b I measured in Part 3 is the second kind: most of it sat on the CPU and it still held 14 tok/s. That is thanks to a design that only keeps the actively working portion small (MoE), which leaves usable speed even after it overflows.
Wrap-up
When you are picking a local LLM for a 12GB card like the Intel Arc B580, looking at the size of the weights is not enough. Depending on the context length you actually use and how thick that model’s working memory is, a model that ought to fit can spill onto the CPU and run several times slower.
The practical fix is simple. With Ollama, trimming num_ctx to the length you actually need can be enough to keep everything on the GPU. The same qwen3:14b ran comfortably at 36 tok/s with the context set to 2,048. And if you genuinely need a long context, choosing a model that keeps its working memory small — the gemma4 family, for instance — is a decision you can make deliberately.
The numbers here are from July 2026, on this setup: a B580 seated directly in a desktop slot, Mesa 26.1.5, Ollama.
Sources
Hardware used in this test
As an Amazon Associate we earn from qualifying purchases.