Is Bigger Smarter? Testing Local LLM Reasoning with 8 Puzzles on the GMKtec EVO-X2

2026年7月8日

Last time, the topic was “speed." Once you know the memory bandwidth, you can roughly predict how fast a model will generate text. So suppose you can fit a large model onto a mini-PC with lots of memory — what about the thing that actually matters, the content itself? In other words, how smart is it? Can we flatly say that bigger is smarter? I had a whole lineup of models solve the same set of reasoning puzzles, and checked for myself. (Measured on actual hardware, as of June 2026.)

How I measured it: eight trick questions, all under the same conditions

What I used was eight slightly mean-spirited reasoning puzzles — the kind that, a generation ago, even large models used to trip over. I mixed in “gotcha" arithmetic like the bat-and-ball price problem, two-step percentage calculations, counting the number of letters in a word, day-of-the-week calculations, and so on. Since every one of these has a single fixed answer, right or wrong can be scored mechanically. To keep things from wobbling, I fixed the temperature (the randomness setting) at zero. On the EVO-X2, I had everything from a small 8-billion-parameter model (parameters being the internal weights a model learns — B stands for billion) up to a huge 235-billion-parameter model solve the same questions.

Results: a 9-gigabyte model tied the 235-billion giant

Line up the number of correct answers, and it did not come out to a tidy “bigger is smarter."

Among the perfect scores sits a 14B model that weighs just 9.3 gigabytes. Meanwhile, the base 235-billion model — which eats up more than ten times the space — fell one step short of a perfect score. The ranking by size and the ranking by smarts do not line up neatly. (For reference: MoE, or “mixture of experts," is a design where only a fraction of the model’s parameters activate for any given input, so a model can be large on disk yet run comparatively fast — the “3B active" note means only about 3 billion of the parameters fire per token. “Tokens" are the chunks of text a model reads and writes, so tokens/sec is roughly its typing speed.)

What I learned (part one): what matters isn’t size, but “does it write out its thinking"

What separated the perfect scorers from the ones that dropped a question? It wasn’t capacity, and it wasn’t quantization (the compression ratio used to shrink the model). The one question they missed was, in almost every case, the multi-step “multiply the percentage in two stages" problem. Stop at the first stage and you get it wrong. Every model that scored full marks has a “chain-of-thought" character — it writes out the intermediate calculation before giving its answer (chain-of-thought meaning the model reasons step by step in the open rather than jumping straight to a conclusion). Conversely, the instant-answer 70B tried to produce the answer without writing out its thinking, and slipped up right here. For this kind of problem, taking the extra step of “reasoning through it in order" was a shortcut compared to simply adding more size.

That said, let me set down an honest caveat here. Eight questions is a narrow yardstick, and it has a low ceiling — a good reasoning-type model reaches a perfect score almost immediately. “The 14B and the 235-billion tied" only means that both cleared this particular set of problems; it does not mean their overall ability — the breadth of their knowledge, their handling of much more involved tasks — has drawn level. Make the problems harder, and the raw strength of the larger-capacity models should start to show. The accurate reading is simply this: “for reasoning at this level, even a small model does fine as long as it’s a reasoning type."

What I learned (part two): raising the compression quality doesn’t bring the smarts back

When you run a huge model on your own hardware, you load it in compressed (quantized) form. Thinking “maybe it would get smarter if I compressed it more carefully," I measured the same 235-billion model at two levels — roughly 2-bit and roughly 3-bit — and compared. The result: both got seven questions right, and they missed the same question. So the cause wasn’t the coarseness of the compression; it was the model’s own innate character. Even compressing more carefully, at the cost of a 20% larger footprint, did not bring that one question back.

An honest aside: it was the scoring program that was wrong

Let me confess one thing. In my first tally, it looked as though every model had missed the trick question whose answer is “you can make it in five minutes." Something seemed off, so I opened up the internals — and the cause was on the scoring side. The question tells the model to “answer with a number only," yet the scoring program was marking anything that didn’t contain the word “5 minutes" as incorrect. The models had, as instructed, correctly answered just “5." Once I fixed the scoring condition, every model went up by one point across the board. It was a case that drove home how doubting your own yardstick is part of measuring, too.

Add a reasoning type, and the 235-billion reaches a perfect score too

I had the multi-step percentage calculation that the base giant model dropped solved by a “reasoning-type" model in the same family. This one writes out the procedure one step at a time before drawing a conclusion, so it didn’t skip the second-stage calculation and got it right without trouble — a clean sweep of all eight questions. Rather than saying the sheer size did the trick, it’s more accurate to say that changing the “way of thinking" is what got it there. Note, though, that because it writes out a long stretch of reasoning, each response takes more time. Smarts come at the cost of a longer wait. That’s the trade-off here.

How closely does it keep pace with the latest cloud models?

From here on, this is not measurement but positioning, based on the published figures from various benchmarks (research, as of June 2026). According to the model cards OpenAI has published and similar sources, gpt-oss 120B reaches roughly 90% on the index that measures general knowledge and reasoning together (MMLU), sits at a level that nearly matches a paid cloud small-but-high-end model (o4-mini) on major reasoning tasks, and on some tasks surpasses the former flagship, GPT-4o. There’s still a gap to the very latest, very top-tier frontier models, but it seems fair to say that “smarts approaching a slightly older cloud upper tier now run on a mini-PC on your desk." In terms of the balance of capacity, speed, and smarts, my hands-on impression after measuring was that rather than going to the trouble of loading the 235-billion, gpt-oss 120B — which scores full marks at half the footprint — is the easier large model to handle for everyday use.

Conclusion: “being able to load it" and “being smart" are two different things

Being able to load a large model in the first place is a strength of a mini-PC’s unified memory. But being able to load it and being smart did not necessarily go together. What sets smarts apart is not capacity or compression ratio, but whether the model “reasons through things in order." If that’s the case, then when picking a single large model for your own use, gpt-oss 120B — full marks, reasonably fast, and modest in footprint — is the front-runner to start with. The huge 235-billion is a trump card for taking on harder problems with a reasoning type, to be called up only when you actually need it. That’s the way of dividing the work that emerged from what I measured.

References

• OpenAI, “Introducing gpt-oss" (model card and benchmark figures)
• gpt-oss-120b & gpt-oss-20b Model Card (arXiv)

* The results for the eight reasoning puzzles are my own measurements on the EVO-X2 (June 2026). The comparison with the latest cloud models is a positioning based on the published benchmarks above.