Six Local LLMs, Same Prompts: Speed, Japanese Output, Summarizing and Code

2026年8月7日

I run local LLMs on a machine with an RTX 3090 and an RTX 3060. The number of models available in Ollama keeps growing, and it has become genuinely hard to answer the question “so which one should I actually use?"

As of April 2026 the main families are Qwen3 (Alibaba Cloud), Gemma4 (Google DeepMind), Llama (Meta), Phi (Microsoft) and Mistral (Mistral AI). So I narrowed it to six models that run easily in Ollama, gave them all the same prompts, and compared speed, Japanese output quality, summarization and code generation.

The short version: for everyday use gemma4 is both the fastest and the most natural in Japanese. For serious reasoning, qwen3:14b has the best balance. And there is a trap: reasoning models can return no answer at all if one setting is wrong.

All measurements are from April 2026.

A note for readers outside Japan: this comparison uses Japanese prompts, so the “language quality" axis is about Japanese output. If you are choosing a local model for a non-English language, that is exactly the axis that is hard to find data on — which is why I have kept it here rather than swapping in English prompts.

Test environment

Item	Specification
GPU 1	NVIDIA RTX 3090 (24GB VRAM)
GPU 2	NVIDIA RTX 3060 (12GB VRAM)
Inference engine	Ollama 0.20.2
OS	Ubuntu 24.04 LTS

The six models

Model	Parameters	Type	Character
qwen3:8b	8B	Standard	Alibaba Cloud. Fast and light
qwen3:14b	14B	Standard	The step up from 8B. Balanced
qwen3.5:9b	9B	Reasoning	Runs an internal chain of thought
qwen3.5:27b	27B	Reasoning	Accurate but heavy
gemma4 (E4B)	8B	Standard	Google DeepMind. Natural Japanese
gemma4:26b	26B	Reasoning	Reasoning, but with practical thinking time

The classification is based on how they behaved in this test. The qwen3 family and gemma4 (E4B, 4B effective) are hybrids that can switch thinking on and off; here they answered immediately, so I have listed them as standard.

A “reasoning model" runs an internal reasoning process before producing an answer — the way a person thinks before speaking. Accuracy goes up, at the cost of time and tokens.

Test 1: Japanese generation — “describe the four seasons of Japan in 100 characters"

qwen3:8b — 121.7 tok/s

日本は四季分明。春（3〜5月）に桜、夏（6〜8月）に梅雨と熱帯夜、秋（9〜11月）に紅葉、冬（12〜2月）に雪国。気候の違いで地域ごとに風景が変わる。

The fastest at 121.7 tok/s. But it mixes in vocabulary that reads as Chinese-influenced (“四季分明", “雪国" used oddly). Readable as Japanese, but stiff.

qwen3:14b — 48.6 tok/s

日本の四季は、春の桜、夏の祭り、秋の紅葉、冬の雪で知られる。

Less than half the speed of the 8B, but the Japanese is clean. A little terse, yet accurate and natural.

qwen3.5:9b — 45.8 tok/s

No output (used the whole 4096-token budget thinking, never reached an answer)

This was the biggest finding of the test. qwen3.5:9b is a reasoning model, so it thinks before answering. With the token limit (num_predict) at 4096, it spent the entire budget thinking and produced no answer at all. More on this below.

qwen3.5:27b — 25.4 tok/s

No output (same cause)

The same problem at 27B. Model size does not change the constraint.

gemma4 — 114.0 tok/s

春は穏やかな訪れ、桜が舞い散ります。夏は生命力あふれる緑と陽射し。秋は実りの豊かさと紅葉の彩り。冬は雪化粧の静けさの中、次の春を待ちます。

114.0 tok/s from an E4B (4B effective) model, and the Japanese is literary — phrases like “桜が舞い散ります" (cherry blossoms dance and fall) and “雪化粧の静けさ" (the quiet of a snow-dressed landscape) read like a person wrote them. Google DeepMind’s Japanese tuning shows.

gemma4:26b — 95.0 tok/s

日本には、春、夏、秋、冬の明確な四季があります。春は桜、夏は暑さと祭り、秋は鮮やかな紅葉、冬は雪景色と、季節ごとに美しい自然の移ろいを楽しめます。

A 26B reasoning model — but this one finished thinking in 2,035 tokens and produced its answer properly. That is the crucial difference from the qwen3.5 family. Accurate, readable Japanese, and 95.0 tok/s is plenty fast.

Test 1 summary

Model	Speed	Japanese quality	Answered?
gemma4	114.0 tok/s	◎ literary, natural	Yes
gemma4:26b	95.0 tok/s	◎ accurate, readable	Yes
qwen3:8b	121.7 tok/s	△ somewhat stiff	Yes
qwen3:14b	48.6 tok/s	○ terse, accurate	Yes
qwen3.5:9b	45.8 tok/s	— no output	No
qwen3.5:27b	25.4 tok/s	— no output	No

Test 2: Summarization — “reduce this technical document to three lines"

I gave each model an English technical explanation of Ollama’s GPU settings (about 800 characters) and asked for a summary in three lines — in Japanese, from an English source.

The source covers CUDA/ROCm/Metal detection, CPU fallback when VRAM is short, manual control through the OLLAMA_NUM_GPU environment variable, and behaviour with multiple GPUs.

qwen3:8b produced an accurate, tight three-line summary, but left technical terms like “fallback" untranslated — a little unfriendly for beginners.

qwen3:14b showed the headroom of the larger model: it rephrased “fallback" into plain Japanese (“switches over automatically") without losing any of the content.

qwen3.5:9b / qwen3.5:27b produced no output again, with num_predict still at 4096. Even for a short summary, a long thinking process never reaches the answer.

gemma4 was high on all three counts — accuracy, natural Japanese, brevity — and its addition of “parallel processing" showed it had correctly understood the English source.

gemma4:26b gave the most detailed summary, including the precise statement that “layers of the model are distributed across each GPU". The reasoning paid off.

Model	Accuracy	Brevity	Japanese	Answered?
gemma4:26b	◎	○	◎	Yes
gemma4	◎	◎	◎	Yes
qwen3:14b	◎	◎	○	Yes
qwen3:8b	○	◎	△	Yes
qwen3.5:9b	—	—	—	No
qwen3.5:27b	—	—	—	No

Test 3: Code generation — “write a recursive Fibonacci in Python"

The prompt asked for a recursive implementation with a docstring and comments.

qwen3:8b produced working code with an English docstring and no comments — the minimum implementation, ignoring the “with comments" part of the instruction.

qwen3:14b delivered type hints, a Japanese docstring and appropriate comments, satisfying every part of the instruction.

qwen3.5:9b / qwen3.5:27b: I re-ran these with num_predict raised to 8192, and they produced their answers properly — careful implementations including a note on time complexity (O(2^n)), a recommendation to memoize, and input validation raising ValueError. The 27B additionally appended a memoized version.

def fibonacci(n: int) -> int:
    """
    Compute the n-th Fibonacci number recursively.
    Time complexity: O(2^n) — exponential. Not recommended for large n.
    Consider memoization (functools.lru_cache).
    """
    if n < 0:
        raise ValueError("n must be a non-negative integer")
    if n <= 1:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

gemma4 met every part of the instruction — type hints, Japanese docstring, comments — which is impressive at 114.0 tok/s.

gemma4:26b gave the most complete output of any model: a worked example of the sequence, a note on complexity, error handling, and an f-string in the error message.

Model	Runs?	Docstring	Comments	Extras
gemma4:26b	Yes	◎ detailed	○	◎ complexity + error handling
qwen3.5:9b (8192)	Yes	◎ detailed	○	◎ complexity + error handling
qwen3:14b	Yes	◎	○	○
gemma4	Yes	◎	○	○
qwen3:8b	Yes	△ English only	×	×

Reasoning pays off in code generation — but the qwen3.5 family requires you to adjust num_predict first. Counting “works out of the box" as part of the score, gemma4:26b is the most practical.

Family comparison

Speed

qwen3:8b is fastest, with gemma4 essentially level at 114.0 tok/s. Both feel instant; you cannot tell them apart in use.

gemma4:26b at 95.0 tok/s is unusually quick for a 26B model — it seems to make good use of the RTX 3090’s memory bandwidth (936 GB/s).

The qwen3.5 family is slower still in practice, because the thinking process is added on top. 25.4 tok/s feels like “a short wait", and the thinking time makes the real wait longer than that number suggests.

The reasoning-model trap: get num_predict wrong and no answer comes out

This was the most important finding of the whole test.

Reasoning models in the qwen3.5 family can fail to produce any answer if the token limit is set low.

A reasoning model runs an internal chain of thought before answering, and those thinking tokens count as ordinary tokens. In this test the maximum output tokens per response (num_predict) was set to 4096, matching the default context length. For qwen3.5:9b and qwen3.5:27b, even “describe the four seasons in 100 characters" consumed the whole 4096 in thinking.

What happened:
  [thinking: 4096 tokens] → hit the num_predict ceiling → no answer

What should happen:
  [thinking: 2000] + [answer: 200] = 2200 total → answer produced

The fix is simply to raise num_predict.

# In a Modelfile
PARAMETER num_predict 8192

# Or per API request
curl http://localhost:11434/api/generate -d '{
  "model": "qwen3.5:9b",
  "prompt": "...",
  "options": { "num_predict": 8192 }
}'

Note that raising it also lets reasoning models think for longer than they need to. If you have VRAM to spare, 8192–16384 and letting the model decide when to stop is the practical setting.

gemma4:26b is also a reasoning model, but it finished thinking in 2,035 tokens and answered properly. Same category, very different efficiency. That is a clear advantage for gemma4:26b.

In the code test, qwen3.5:9b with num_predict at 8192 produced high-quality code. Reasoning models are strong once configured — and useless if configured wrong.

VRAM use and which GPU you need

Model	Parameters	VRAM (approx.)	RTX 3060 (12GB)	RTX 3090 (24GB)
qwen3:8b	8B	≈5.5GB	Fits easily	Yes
gemma4 (E4B)	8B	≈6.0GB	Fits easily	Yes
qwen3.5:9b	9B	≈7.9GB	Just fits	Yes
qwen3:14b	14B	≈9.5GB	△ nothing else alongside	Yes
gemma4:26b	26B	≈17.0GB	× does not fit	Yes
qwen3.5:27b	27B	≈17.4GB	× does not fit	Yes

VRAM figures are estimates based on model file size. Actual use varies with context length (the KV cache) and implementation.

The 8B class fits in 5–8GB, so an RTX 3060 (12GB) handles it comfortably. With two cards, keeping an 8B model resident on the 3060 for chat while running a 26–27B model on the 3090 only when needed is an efficient arrangement.

Models of 26B and up need around 17GB, so an RTX 3090 or 4090 class card (24GB) is required. They physically do not fit on a 12GB card.

Which one should you use?

If you are unsure, start with gemma4. 114 tok/s with literary Japanese makes it ideal both as an introduction to local LLMs and for daily use.

When you want more accuracy, try qwen3:14b. The speed halves, but 48.6 tok/s is still comfortable.

If you want reasoning, gemma4:26b — it manages 95.0 tok/s while thinking, and has none of the num_predict trap. Worth trying if you own an RTX 3090.

Wrap-up

Speed: qwen3:8b (121.7 tok/s) and gemma4 (114.0 tok/s) lead. Both feel instant.

Japanese quality: gemma4 did unexpectedly well. Getting literary phrasing out of an 8B model was a surprise.

Summarizing and code: gemma4:26b is the most consistent overall — reasoning accuracy without configuration hassle. qwen3:14b is a strong all-rounder.

The reasoning trap: the qwen3.5 family may produce nothing with num_predict around 4096. gemma4:26b, the same category, has no such problem. That difference matters.

Best balance: qwen3:14b. High on speed, accuracy and Japanese quality alike. The one to run as your main model.

Hardware used

As an Amazon Associate we earn from qualifying purchases.