How Fast Are Models That Fit in 12GB? Speed and Power Measured | Intel Arc B580 Local LLM Part 3

2026年8月4日

After the driver update in Part 1 turned the Intel Arc B580 from unusable into something 3.3x faster, I wanted to know what the card actually does at its normal speed. So I ran the same test across models of different sizes.

The short version: the B580 reads prompts (prefill) very fast, but generating text (decode) runs into the memory bandwidth ceiling and falls off as the model grows. And it draws 34W even when it is doing nothing.

Everything here was measured on a B580 seated directly in a desktop PCIe slot, running Ollama on Vulkan (Mesa 26.1.5). This part only covers models that fit inside the 12GB of VRAM. As of July 2026.

The previous article is here

The machine

I added the Intel Arc B580 to the desktop below.

What I wanted to know: how far does 12GB get you?

With local LLMs, a bigger model is smarter but slower and heavier. What I wanted was a sense of where the comfortable limit sits on a 12GB card.

So I worked up in steps, from a small 2.4GB model to a mid-size 7GB one that only just fits.

What I measured: decode, prefill, and power draw

For each model I recorded the following.

• decode (how fast text is generated, tok/s) — higher is faster
• prefill (how fast the input prompt is read, tok/s) — this is what matters when you feed it long documents
• average power draw during generation (W), and the efficiency derived from it (tok/s per W)

Every run used the same prompt (about 1,000 tokens), three times, and I took the median. Before each run I confirmed that no inference process was sitting on the RTX 3090 in the same machine, so that the work really was landing on the B580. That check comes from getting it wrong in Part 1.

Results: smaller is faster, and prefill stands out

Five models that fit in 12GB, ordered by VRAM footprint.

Here is the decode side as a chart.

[kimono_bar title="Arc B580 decode speed (tok/s, higher is faster)" unit="tok/s" highlight="2″ note="Ordered by VRAM footprint. Decode falls as the model grows — this is a memory bandwidth limit. Measured July 2026."]
phi4-mini (2.4GB) | 79.81
nemotron-3-nano:4b (2.4GB) | 95.52
qwen3:8b (4.5GB) | 60.08
Ornith-1.0-9B (4.7GB) | 48.03
gemma4-12b (6.9GB) | 29.95
[/kimono_bar]

The clear pattern is that decode drops as the model grows. The 2.4GB models ran at 80–95 tok/s; the 6.9GB gemma4-12b came in at 29.95 tok/s. Generating text is bound by memory bandwidth — the width of the pipe the data travels through — and a bigger model means more data read for every single token.

Prefill, on the other hand, was striking on the small models. 2,380 tok/s on phi4-mini and 1,651 tok/s on nemotron-3-nano:4b are solid numbers for a card in this class. If your work involves pushing long prompts through in one go, that speed is what you feel.

A model that “should fit" but slowed down anyway (an honest note)

I also measured one dense 14B model (qwen3:14b, 5.9GB of weights). Its decode came out at 9.14 tok/s — below the larger gemma4-12b at 29.95. Even though the weights themselves fit inside 12GB.

The likely cause is that its default context length (how much input it can hold at once) is large, and once the working memory for that — the KV cache — is added, the total exceeds 12GB, so the overflow gets handled on the CPU side. My placement check only confirmed that the work was not on the RTX 3090; it could not detect a spill to the CPU. I am recording it as a caution: on a 12GB card, the effective speed depends not only on the size of the weights but on the context length setting. I dig into this in the next part.

Power: 34W at idle is the weak point

I measured power draw too. During generation it sat between 74W and 133W. In efficiency terms the small models were good, at roughly 1 tok/s per watt, while gemma4-12b dropped to 0.22. The heavier the model, the worse both speed and efficiency get.

The weak point is idle. Doing nothing at all, the card still drew 34W. For a GPU you leave permanently attached to a mini PC, that standby draw is not something you can ignore. It suits a setup where you leave it unloaded when unused, or unplug it entirely.

Where the B580 fits, and where it does not

Putting the measurements together, the shape of the card comes out fairly clearly.

As Part 1 showed, this lands in the same class as an RTX 3060. As a 12GB card that is easy to buy new, it is a reasonable choice for running small and mid-size models.

Wrap-up

For models that fit inside 12GB, the Intel Arc B580 is practical enough. Reading prompts is its strength, which suits long-input work. Text generation falls off as models grow because of the bandwidth limit, and the 34W idle draw is the weak spot.

What happens if you push past 12GB anyway is the subject of the next part, once the measurement method is tightened up. The numbers here are from July 2026, on this setup: a B580 seated directly in a desktop slot, Mesa 26.1.5.

Sources

Hardware used in this test