What Runs on 8GB? I Measured 22 Local LLMs and Sorted Them by Memory Tier

September 22, 2026

Running local LLMs, I keep hitting the same wall: “will this model fit in my memory?"

What a model’s page shows you is file size — not the memory it actually needs to run. I took the 22 models sitting on my machine and sorted them by how much memory they actually need.

What This Article Covers, and How I Measured

Three things: how required memory differs from file size, what runs at each memory tier, and what you actually gain by adding more memory.

Speed numbers were all measured under the same conditions.

Machine	GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB unified memory)
GPU	Integrated Radeon 8060S
Runtime (A)	llama.cpp (build b10605), Vulkan / pp512 and tg128 [tok/s] / measured September 10, 2026
Runtime (B)	ollama (0.32.15), ROCm / ollama’s built-in timer [tok/s] / measured September 21, 2026
Runs	5 runs per condition. Sorted, top and bottom dropped, median of the middle 3 (for B, the first run — which includes loading — is discarded before those 5)

I used the integrated GPU because it’s the only place I can measure all of them under the same conditions. My discrete graphics cards don’t have enough capacity — the larger models simply don’t fit.

There are two runtimes because llama.cpp could not load a number of these models. They share one table, but read A and B as separate columns. The backends differ (Vulkan vs ROCm) and so does the timing method. You cannot add or average a number from A with one from B. In practice B comes out roughly 1.8x higher than A on the same model.

Required Memory Is a Bit More Than File Size

Here’s the rule of thumb that came out of my own measurements.

Required memory ≈ file size + about 0.7GiB

A concrete example: Qwen3.8-27B at UD-Q4_K_M is a 15.33GiB file. Add the overhead, and the actual requirement came to 16,397MiB.

A 16GB graphics card has 16,304MiB usable. It fell short by about 100MiB, and did not fit.

A 16GB model does not fit on a 16GB card. Comparing the catalog numbers alone makes it look like it should fit, and it doesn’t. Add this overhead before making the call.

From 8GB to 128GB — 22 Models, Measured Two Ways

Memory needed	Model	Released	Parameters	Quantisation	Architecture	Size	A — llama.cpp / Vulkan		B — ollama / ROCm
							Read	Write	Read	Write
8GB	llama3.2:1b	2024/09	1.2B	Q8_0	Dense	1.23GB	7,107.3	158.0	21,539.3	253.2
	phi4-mini	2025/02	3.8B	Q4_K_M	Dense	2.32GB	2,368.4	79.7	6,965.7	140.0
	nemotron-3-nano:4b	2026/03	4.0B	Q4_K_M	Hybrid	2.64GB	1,392.6	69.5	4,178.2	132.3
	llama2:7b	2023/07	7B	Q4_0	Dense	3.56GB	—	—	2,247.9	71.6
	qwen3:8b	2025/04	8.2B	Q4_K_M	Dense	4.87GB	1,191.0	41.0	3,496.2	76.9
	llama3.1:8b	2024/07	8.0B	Q4_K_M	Dense	4.58GB	—	—	2,169.2	64.0
	Ornith-1.0-9B	2026/06	8.95B	Q4_K_M	Dense	5.24GB	1,073.8	39.1	2,403.0	72.8
16GB	qwen3:14b	2025/04	14.8B	Q4_K_M	Dense	8.64GB	728.5	24.5	1,900.5	44.0
24GB	qwen3.6:27b	2026/04	27.8B	Q4_K_M	Hybrid	17.0GB	—	—	275.0	12.1
	qwen3-coder:30b	2025/07	30.5B	Q4_K_M	MoE	17.28GB	1,211.9	92.4	894.6	59.1
	gemma4:31b	2026/03	31.3B	Q4_K_M	Dense	19.0GB	—	—	248.8	10.1
	glm-4.7-flash	2026/01	29.9B	Q4_K_M	MoE	19.0GB	—	—	698.9	50.5
	Ornith-1.0-35B	2026/06	34.7B	Q4_K_M	MoE	19.71GB	1,073.9	75.6	828.7	57.9
32GB	nemotron-3-nano:30b	2025/12	31.6B	Q4_K_M	Hybrid MoE	22.60GB	921.4	69.6	741.5	64.0
	qwen3.6:35b-a3b	2026/04	36.0B	Q4_K_M	MoE	23.0GB	—	—	770.8	52.1
64GB	llama3.3:70b	2024/11	70.6B	Q4_K_M	Dense	39.60GB	102.0	5.3	122.7	5.0
	qwen3-coder-next	2026/01	79.7B	Q4_K_M	MoE	51.0GB	—	—	489.8	47.6
128GB	gpt-oss:120b	2025/08	116.8B	MXFP4	MoE	65.0GB	—	—	356.2	35.1
	GLM-4.5-Air	2025/07	110.5B	Q4_K_M	MoE	67.85GB	276.9	25.6	225.1	22.3
	qwen3-235b	2025/04	235.1B	Q2_K	MoE	79.81GB	159.4	21.8	106.4	17.9
	nemotron-3-super:120b-a12b	2026/03	123.6B	Q4_K_M	MoE	86.0GB	—	—	202.0	20.4
	minimax-m2.7	2026/04	228.7B	IQ4_XS	MoE	100.97GB	201.4	29.1	39.7	20.1

The dashes in column A are models llama.cpp could not load. Not a capacity problem, and not a corrupted file. I go through the reasons in a later section. Column B has a number for every row.

llama2:7b and llama3.1:8b were added later, to compare two generations from the same vendor. Column A is empty for those two because I only measured them the B way.

I included release dates because whether something runs and whether there’s a reason to pick it today are two different questions.

Why Do Same-Size Models Run at Different Speeds?

Line up models of almost the same size, and only the architecture is left to explain the gap. All four below were measured on the same day, the B way (ollama / ROCm).

Model	Size	Architecture	Write speed [tok/s]
gemma4:31b	19.0GB	Dense	10.1
Ornith-1.0-35B	19.71GB	MoE	57.9
qwen3.6:27b	17.0GB	Hybrid	12.1
qwen3-coder:30b	17.28GB	MoE	59.1

5.7x between the two around 19GB, and 4.9x between the two around 17GB. They take up the same space, and the writing speed is this far apart.

The reason is how much has to be read to write each token.

A dense model reads every single weight, every time. A 70B dense model pushes all 39.6GB through on every token. MoE is different — it’s internally split into sections, and only reads the section responsible for that token. minimax-m2.7 has 228.7B parameters total, but only a slice of that runs on any given pass.

Generation speed is roughly determined by this division:

Write speed ≈ memory bandwidth ÷ amount read per token

A MoE model with a small denominator runs fast even with a large total size. But fitting it still requires the full total. A 101GB model, however fast it runs, still needs 101GB of space to sit in.

Where it fits is decided by total size; how fast it runs is decided by how much gets activated per token. Pick your memory based on the total; judge speed based on the active portion.

Does a Model’s Release Date Tell You Anything About Speed?

Looking down the release-date column, one question came up: does a year of difference change the speed? Comparing across vendors mixes in design differences, so I lined up two llama models from Meta instead. Both measured the B way.

Model	Released	Size	Architecture	Write speed [tok/s]
llama2:7b	2023/07	3.56GB	Dense	71.6
llama3.1:8b	2024/07	4.58GB	Dense	64.0
llama3.3:70b	2024/11	39.60GB	Dense	5.0

A year apart, and the speed barely moved. The newer llama3.1 looks slightly slower only because its file is 1GB larger. Converted to how much is read per second, that is 255GB against 293GB — the newer one is actually reading more. Either way, it is a small shift for a year’s gap.

So what is moving the speed? The table above answers it: the architecture, not the year. Still from Meta, llama3.3:70b is a 39.60GB dense model and drops to 5.0 tok/s. Meanwhile nemotron-3-nano:30b, released December 2025, is a 22.60GB hybrid MoE at 64.0 tok/s. A little over half the size, and 12.8x the speed.

In other words, it is not that “newer is faster" — it is that MoE has become more common among newer models. If the year itself were doing the work, llama2 and llama3.1 would have shown a gap. They did not.

I did not measure how clever any of them are, so that side is out of scope. If you are picking on space and speed alone, the column to read is the architecture, not the release date.

Why Couldn’t llama.cpp Read Them?

The six rows showing a dash in column A are models llama.cpp failed to load. I first tried build b10605, then b10941. Same result either way.

It is not a capacity problem. The smallest of them, qwen3.6:27b, is 17GB and failed with 75GB free. The files are not corrupted either.

The way each one stopped, though, was different. These are the messages from b10941.

Model	Where it stopped	Kind
gpt-oss:120b	Architecture not recognised (unknown model architecture: 'gptoss')	Unsupported
glm-4.7-flash	Same — not recognised ('glm4moelite')	Unsupported
qwen3.6:27b / 35b-a3b	A settings array is the wrong length (expected 4, got 3)	Mismatch
gemma4:31b	Wrong number of tensors (expected 1189, got 833)	Mismatch
qwen3-coder-next	A tensor that should be there is missing (blk.0.ssm_dt.bias)	Mismatch
nemotron-3-super:120b-a12b	A tensor has the wrong shape	Mismatch

That is the dividing line. Only two of them — gpt-oss:120b and glm-4.7-flash — are models llama.cpp does not know about at all. For the other four kinds, llama.cpp does have the code to read them. They still fail because the files ollama distributes and the shape llama.cpp expects do not line up.

So “llama.cpp hasn’t caught up with new models" does not cover it. There are models it has not caught up with, and there are models where both sides are current but the pieces do not fit. The second kind will not be fixed by waiting on one side alone.

Feed the same files through ollama, and all six run. That is why column B has numbers for them. When something will not load, it is worth swapping the runtime before giving up on the model.

From 192GB to 2TB — Sorted by the Publisher’s File Size

Past this point, I can’t run anything myself. The speed column is blank. Sizes are counted from the files published on Hugging Face (as of September 10, 2026).

Memory tier	Model	Released	Total parameters	Quantisation	Size	Speed
192GB	DeepSeek V4 Flash	2026/04	—	UD-Q8_K_XL	161.9GB	—
	MiniMax M2.7	2026/04	228.7B	UD-Q6_K_XL	207.4GB	—
256GB	GLM-5.2 / 5.3	2026/06	744B	UD-IQ1_S	216.7GB	—
	MiniMax M2.7	2026/04	228.7B	Q8_0	243.1GB	—
512GB	GLM-5.2 / 5.3	2026/06	744B	UD-Q4_K_XL	467.3GB	—
	Kimi K3	2026/06	2.8T	UD-Q1_0	466.4GB	—
	MiniMax M2.7	2026/04	228.7B	BF16	457.5GB	—
1TB	GLM-5.2 / 5.3	2026/06	744B	Q8_0	801.4GB	—
	Kimi K3	2026/06	2.8T	UD-Q2_K_XL	861.3GB	—
	DeepSeek V4 Pro	2026/04	~1.6T	UD-Q8_K_XL	873.4GB	—
2TB	Qwen3.8-2.4T	2026/08	2.4T	UD-IQ4_XS	1,310.9GB	—
	GLM-5.2 / 5.3	2026/06	744B	BF16	1,508.0GB	—
	Kimi K3	2026/06	2.8T	UD-Q8_K_XL	1,561.2GB	—
Doesn’t fit even in 2TB	Kimi K2.6	2026/04	—	BF16	2,054.1GB	—
	Qwen3.8-2.4T	2026/08	2.4T	BF16	4,893.2GB	—

Notice that the parameter-count ordering and the size ordering don’t match here. Kimi K3 is 2.8T parameters and Qwen3.8-2.4T is 2.4T, but at native precision Qwen3.8’s size is more than 3x larger.

Download size depends on what precision the publisher chose to release, not parameter count alone. You can’t plan storage from the parameter count by itself.

Who Actually Needs 512GB?

The difference between 128GB and 512GB isn’t the number of models that run. It’s whether two additional top-tier models become reachable.

At 128GB, DeepSeek V4 Flash, MiniMax M2.7, and Qwen3-235B all run. What doesn’t is GLM-5.2 and Kimi K3 — both, even at their most aggressively cut 1-bit-class quantisation, still weigh in at 216.7GB and 466.4GB. Even fully stripped down, they don’t fit.

At 512GB, GLM-5.2 fits at Q4 (467.3GB). You no longer have to cut it all the way down to 1-bit. Kimi K3 also just barely fits at 1-bit.

Whether you actually want to touch these two specific models is what decides whether 512GB is worth it for you. For any other purpose, 128GB pretty much covers it.

What Changes If You Scale to 2 or 4 Machines?

Connect two 512GB machines into a 1TB cluster, and Kimi K3 moves from 1-bit up to 2-bit (861.3GB), while GLM-5.2 climbs to Q8 (801.4GB) — placeable with essentially no quality loss.

Scale to four machines for 2TB, and GLM-5.2 runs at BF16 (1,508.0GB) — the precision the publisher released it at, untouched. Kimi K3 also reaches Q8 (1,561.2GB).

What adding more machines buys you isn’t speed. It’s getting to step away from quantisation. With one or two machines, a large model always means cutting something down to make it fit. With 2TB, some models can be placed without cutting anything at all.

Speed is a separate matter, though. AMD has a published record of connecting four machines with the same chip over 5Gbps wired Ethernet and running a trillion-parameter model at 9.45 tok/s. Connect them and it runs, but crossing machine boundaries costs you speed. Going to four machines doesn’t get you 4x the speed.

What I Haven’t Confirmed in This Article

• I haven’t measured capability. This article only looks at footprint and speed
• I have not run a single model larger than 128GB
• The roughly 0.7GiB overhead is a rule of thumb derived from Qwen3.8-27B’s quantisation steps. It will vary by model and configuration
• Speed was measured on one machine with one GPU. A different machine will produce different numbers
• I tried two builds of llama.cpp, b10605 and b10941. A newer build might work, but I have not tested that
• Columns A and B use different backends (Vulkan and ROCm). I have not separated how much of the gap comes from the backend and how much from the software itself
• Column B comes from ollama’s built-in timer, which is not the same basis as llama-bench’s pp512 / tg128
• The 192GB-and-up table is just a tally of publisher file sizes. I have not run a single one of them
• I have not tested a multi-machine cluster myself. The 9.45 tok/s figure comes from AMD’s own announcement

Closing — Memory Is Decided by Total Size, Speed by What Gets Read

The memory a model needs to run is roughly its file size plus about 0.7GiB. A 16GB model does not fit on a 16GB card.

At almost the same size, MoE and dense models differ in speed by more than 5x. A 19.71GB MoE model ran at 57.9 tok/s; a 19.0GB dense model ran at 10.1 tok/s. Where it fits is decided by total size; speed is decided by how much gets read per token.

The release date itself does not do much for speed. Meta’s llama2:7b and llama3.1:8b are a year apart and came in at 71.6 and 64.0 tok/s. What is doing the work is the architecture: a 22.60GB hybrid MoE ran 12.8x faster than a 39.60GB dense model.

The runtime changes the outcome too. All six models llama.cpp could not read ran fine under ollama. And the reason they failed was not only “llama.cpp hasn’t caught up" — some of them are cases where the distributed file and the runtime simply do not fit together.

The two models that don’t run at 128GB are GLM-5.2 and Kimi K3. 512GB exists specifically for these two. Adding more machines lets you step away from quantisation, but it doesn’t make things faster.

Knowing what actually fits in your own memory makes it a lot easier to decide what to add next.