37 Seconds Before the First Character Appears? What I Found by Measuring Longer Prompts

2026年9月11日

I once wrote an article measuring the “wait time" of local LLMs across three machines. That piece left one thing unfinished.

How does the time to the first character stretch out as you make the prompt longer? I’d aimed to measure that, but I couldn’t make the input long enough, and I never got a solid number.

This time I picked that homework back up, and the cause turned out to be in a different place than I expected. Partway through, I had it completely misread.

This is a measurement as of August 2026.

Here’s the previous article.

Test Setup

Machine	GMKtec EVO-X2 (Ryzen AI Max+ 395 / 128GB unified memory)
Model	gpt-oss:120b (65GB, kept resident)
Inference engine	Ollama 0.30.10
Usage	Connected a coding-assistant agent to this local LLM
OS	Ubuntu

0.3 Seconds, or 37 Seconds?

In everything I’d measured up to now, throwing a short question at a model that’s already loaded gets you the first character in 0.29 seconds. That’s the median of five measurements, and the spread was small.

But here’s the number for the same model on the same machine, when I asked it to “read this file" through an agent instead.

How it was sent	Time to first character
Short question (my usual way of measuring)	0.29 seconds
Via an agent, having it read one file	36.7 seconds

That’s 126 times longer. The model stayed loaded the whole time, and I didn’t change machines.

At this point I nearly concluded that “this is how long you wait with an agent, every single time." That was wrong.

Does Every Turn Make You Wait This Long?

Keep the conversation going and measure again, and the picture changes.

	Time to first character
1st turn (cold start)	36.8 seconds
2nd turn onward	2–3 seconds

Across four round trips, the total came to 46.7 seconds. Of that, 36.8 seconds was the first round trip, and the remaining three averaged 3.3 seconds each.

The 37 seconds wasn’t a cost paid every turn — it was a cost paid once, at cold start.

What Was That 37 Seconds Actually Spent On?

Before generating any text, the AI goes through a step where it reads the text it was sent. This is called prefill. In human terms, it’s like reading through a question before you start thinking about the answer.

What surprised me was that all I’d typed was a single line of instruction. Even so, the input came to 33,000 tokens.

Here’s the breakdown.

• The agent’s own instructions (how to behave, what rules to follow)
• A list of the tools it can use (definitions for reading files, writing files, searching, and so on)
• And on top of all that, the one line I actually typed

The length of what the user types is only a tiny fraction of what actually gets sent. Most of it is the description of the tooling.

Here’s what I measured for reading speed. A token is a small unit that text gets broken into; for Japanese, roughly one character works out to 1–2 tokens.

Input length	Reading speed	Time to finish reading
About 4,000 tokens	1,219–1,651 tok/s	2.4–3.3 seconds
About 33,000 tokens	about 900 tok/s	about 37 seconds

An 8x increase in input stretched the wait time by more than 11x. The reading speed itself also dropped as the input got longer.

That said, once this reading is done, it’s remembered. From the second round trip on, only the newly added part needs to be read, which is why it drops to 2–3 seconds.

What Actually Feels Slow in Daily Use?

This was the biggest discovery of this round.

Once you’re past the cold start, reading takes 2–3 seconds. Generation speed is fine too — gpt-oss:120b measured 34.3 tok/s on my machine, with only 3.6% variance. Going by the numbers alone, it should feel comfortable.

And yet each round trip was still taking around 25 seconds.

Digging in, the cause turned out to be how much the model talks.

For an instruction like “keep the duplicate and answer in one line," the answer itself needs only five words. And yet the model used 700–900 tokens to produce it.

gpt-oss is a model that builds up its reasoning before answering. That “thinking" part also counts as generated text and eats up time. Even when you ask for a short answer, it writes a long path to get there.

The slowness you actually feel wasn’t decided by the time spent reading the input, nor by generation speed — it was decided by how much got generated.

What I Had Misread

Let me be honest here. Partway through, I was thinking this:

• The agent resends 33,000 tokens on every turn
• So every move costs 37 seconds
• Ten moves would add up to six minutes

All of it was wrong. The reading happens once, at cold start, and after that it’s 2–3 seconds. Adding more turns doesn’t touch that.

The source of the assumption was that I’d only measured the first turn. I measured once and decided it happened every time. Measuring a few more turns in a row would have shown otherwise.

A Hole I Found in My Own Method

Along the way, I also found a hole in how I’d been measuring.

Ollama has a display, ollama ps, that shows what’s currently loaded. It also reports how long the model took to load. I’d been using this to judge whether things were “ready."

But there’s a state where the model is loaded and it still takes 37 seconds. At that point, ollama ps reports it as loaded, and the load time comes back as nearly zero. Both indicators say “ready," and yet no response comes back.

So I’ve changed two things about how I measure.

First, I stopped using ollama ps to judge “ready." It can’t tell ② apart from the rest.

Second, I now always report the time to the first character alongside everything else. Skip that, and ② and ③ get mixed up.

This also means the reading-speed numbers I’ve published so far come with a condition: they were measured by repeating the same prompt. If you’re feeding it different text every time, you won’t get the same speed. I’ve added a note to the relevant article as well.

What Works With This Setup, and What Doesn’t

Use case	Does it work with this setup?
Short back-and-forth chat	Works fine. Comes back in 0.3 seconds
Having an agent do work (cold start)	Wait 37 seconds, but only on the first turn
Having an agent do work (2nd turn onward)	Reading takes 2–3 seconds. What bottlenecks it is how much it talks

If there’s a fix to try, it’s cutting how much the model says rather than cutting the wait time itself. For a model with a setting that shortens the reasoning it builds up, that’s where it would help. I haven’t tried that yet.

What I Did Not Measure

There’s still more I haven’t measured. What shape does the wait time trace out as you step the input from 4,000 up to 33,000 tokens? This time I only got the two endpoints. And: what would one round trip come to if the reasoning-building part were shortened? Those two are next time’s homework.

The figures in this article are from my own setup (GMKtec EVO-X2 + Ollama 0.30.10, gpt-oss:120b) as of August 2026. Results will differ with a different model or configuration.

In Summary: Which Stage Was the Wait Actually In?

• Even the same machine running the same model can take 0.29 seconds or 36.7 seconds to the first character
• What creates the difference is how long the sent text is: about 2.4–3.3 seconds at roughly 4,000 tokens, about 37 seconds at roughly 33,000 tokens
• But ★that 37 seconds happens once, at cold start — it drops to 2–3 seconds from the second round trip on
• ★What actually caused the felt slowness was the amount generated. 700–900 tokens for an answer that needed five words, adding up to around 25 seconds per round trip
• Generation speed itself, at 34.3 tok/s, is plenty — what’s slow isn’t the “speed," it’s the “length"
• There’s a slow state even while the model is loaded. Neither ollama ps nor the load time can tell you you’re in it

The question I’d flagged as “next time’s homework" in the previous article has an answer, for now. At the same time, I’m left with the lesson that I nearly drew a conclusion from a single measurement.

Hardware Used for This Testing

Here’s the machine used for these measurements.

If you’re loading larger models, storage space runs out before anything else.