Is Waiting for a Long Text Proportional to Its Length? I Measured Time to First Token in Seven Steps of Input

October 14, 2026

14 October 2026

In my previous article I wrote that when you make a local LLM read a long text, the wait before the first character appears gets longer. But all I had then were the two end points. What happens in between I left as homework.

This is that homework: I measured the input length in seven steps. If the length doubles, does the wait simply double?

These are measurements from August 2026.

The previous article (time to first token and prompt length)

Hardware used

The machine used for the measurements is below, the same combination as last time.

Machine	GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB unified memory)
Model	gpt-oss:120b (65GB, kept loaded)
Inference engine	Ollama 0.30.10
Context length	131,072 tokens specified
OS	Ubuntu

What I measured

I measured the time from sending text until the AI’s first character appears on screen. Most of this time is the AI reading the text it was sent.

I stepped the input length in seven steps, from about 1,100 tokens to about 67,600 tokens. A token is a unit into which text is split; in Japanese it is roughly 1 to 2 tokens per character.

How I measured

Measured on	2 August 2026
Runs	Five per step, for seven steps. Sorted, dropped the highest and lowest, and took the median of the middle three
Text	A different text from the very start in each of the five runs. Reusing the same text lets the AI remember what it read last time and skip re-reading
What is measured	The sum of the model’s load time and the time to read the text sent
Settings	Temperature 0, random seed fixed

In the previous article I laid out three states for the time to the first character: (1) the model is not loaded, (2) it is loaded but this conversation is new, (3) the conversation is continuing. What I measure this time is state (2).

What shape appears when the input is stepped in seven?

Here are the results. The spread is the minimum and maximum of the five runs.

Input length	Time to first character	Spread of 5 runs	Reading speed
about 1,100 tokens	1.86 s	1.83-1.95 s	725 tok/s
about 2,200 tokens	3.50 s	3.34-3.52 s	690 tok/s
about 4,300 tokens	6.50 s	6.47-6.60 s	698 tok/s
about 8,500 tokens	13.28 s	13.06-13.37 s	659 tok/s
about 17,000 tokens	30.27 s	30.13-30.44 s	565 tok/s
about 35,000 tokens	85.47 s	84.74-86.21 s	410 tok/s
about 67,600 tokens	313.70 s	312.3-315.4 s	216 tok/s

At about 67,600 tokens, 313.70 seconds is over five minutes. Nothing appears on screen during that time.

The spread of the five runs stayed within 5% of the median at every step. These are not numbers that happened to come out scattered.

If the length is 60 times, is the wait 60 times?

It was not a simple proportion.

Compared	Input	Wait
about 1,100 to about 67,600 tokens	60.1x	169x
about 4,300 to about 35,000 tokens	8.1x	13.2x

In the previous article I wrote, from the two end points, that “when the input is 8 times longer, the wait grows more than 11 times". This time’s 13.2 times for 8.1 times shows the same tendency.

The reason it grows faster than proportion shows at the right edge of the table: the reading speed itself falls. At about 1,100 tokens it was 725 tok/s, and at about 67,600 tokens it was 216 tok/s, less than a third.

The longer the text, the more the AI has to re-work the relationships with the words that came before. The amount to read grows, and the effort per token grows too.

If this reading is right, reading speed should keep falling as the input gets longer. Indeed, of the seven steps, apart from a small recovery from 690 to 698 between about 2,200 and about 4,300, it kept falling to the end.

What if I make it read the same text again?

The measurements above use a different text each of the five times. So what if I send the same text again? As a control I measured this at about 35,000 tokens.

Input of about 35,000 tokens	Time to first character	Runs
Read for the first time	85.47 s	5
Same text again	0.38 s	3 (0.37-0.43 s)

That is one 225th. The AI retains what it has read, so almost no re-reading happens.

In the previous article I wrote that “from the second round trip it drops to 2-3 seconds". That was a value measured during an exchange with an agent. The same thing happens in this measurement with prepared text.

What this says is that the heavy part is sending a long text for the first time. If you keep asking questions with the same material still loaded, you wait only the first time.

Last time’s “37 seconds" and this time’s “85 seconds"

There are places where the numbers do not match the previous article.

	Input	Time to first character	Reading speed
Last time (via an agent)	about 33,000 tokens	about 37 s	about 900 tok/s
This time (prepared text)	about 35,000 tokens	85.47 s	410 tok/s
Last time (via an agent)	about 4,000 tokens	2.4-3.3 s	1,219-1,651 tok/s
This time (prepared text)	about 4,300 tokens	6.50 s	698 tok/s

With the same machine, model and inference engine, there is a gap of more than two times.

Two differences in conditions can be named. Last time’s value was measured during an exchange with an agent, and this time’s with prepared text. And this time I specified a context length of 131,072 tokens. A large context length reserves space for it in advance, which could affect reading speed.

There is also one more candidate. Last time it was measured partway through an exchange, so earlier conversation may already have been sitting in the reserved space. The third measurement in this article is an example of that: sending the same text again took 0.38 seconds even at about 35,000 tokens, because what remains is not re-read.

Last time’s about 900 tok/s is about 2.2 times this time’s 410 tok/s. If about half had already been retained, the gap is roughly explained. But this is a guess, and I have no record of how much remained in last time’s measurement. If this reading is right, measuring only the very first turn of a conversation should have been about as slow last time as this time. If it finished in about 37 seconds even then, the cause lies elsewhere.

I have not separated which of the three candidates is the cause. The effect of context length is also unconfirmed. Measuring again with only the context length changed, and measuring only the first turn of a conversation, are the next two homework items.

The seven steps in this article are one set measured under identical conditions. Steps can be compared with each other as they are, but they are not values to add to or subtract from last article’s numbers.

Why is the time to first character long? Reading and writing phases

LLM processing splits into a phase that reads the text sent (prefill) and a phase that writes the answer one token at a time (decode). The reading phase takes time set by the amount of computation; the writing phase is set by how fast weights are carried from memory. Most of the time to the first character is the reading phase.

In the reading phase, each token in the text computes its relationship with all the tokens before it. As the text grows, each token has more partners, so the effort grows faster than proportion to length. The fall in reading speed from 725 tok/s to 216 tok/s in this article is probably an expression of this property.

What has been read stays in memory as the conversation’s memory (KV cache). That sending the same text again took only 0.38 seconds is because it used this memory as it was and did not re-read.

Three ways to cut the wait

From what I measured, here are three ways to reduce the wait when you make it read long material.

Way	What to do	Basis
Keep asking questions on the same material	Send the material once at the start and only add questions afterward. Do not resend the material	The second time for the same text, 85.47 s to 0.38 s (this article’s measurement)
Don’t unload the model	Ollama by default unloads the model after 5 minutes unused, and the conversation’s memory goes with it. Lengthen keep_alive	Ollama official documentation (keep_alive setting)
Put fixed text at the start	Put always-identical instructions first and the changing part after. If even one character differs at the start, the memory is not used	llama.cpp server’s “reuse only the common prefix" behavior (cache_prompt)

The llama.cpp server has a setting called cache_prompt that skips re-reading for the part whose start matches (on by default). A separate setting called cache-reuse also handles insertions in the middle. Ollama has the same mechanism inside, and while the model is loaded it reuses the memory of input whose start matches. What I confirmed is only the “second time for the same text" on Ollama; I have not measured the effect of the llama.cpp settings.

How reading speed changes with the machine: reference values

Reading-phase speed changes with GPU compute. I line up this blog’s other measurements (short input, pp512) for reference. The model and input length differ from this article’s gpt-oss:120b, so they are reference values to see the tendency.

Machine	Reading speed, Qwen3 8B [tok/s]	Reading speed, Qwen3.6 27B [tok/s]	Measured
RTX 3090 (24GB)	–	590	July 2026
EVO-X2 integrated GPU (Radeon 8060S)	1,237	270	July 2026
Intel Arc B580 (12GB)	1,090-1,193	–	July 2026
Mac mini (M4, 24GB)	623	60	July 2026

Comparing on 27B, the RTX 3090 had about twice the reading speed of the EVO-X2’s integrated GPU, and the Mac mini was about a quarter of the EVO-X2’s integrated GPU (about a tenth of the RTX 3090). How far this article’s 313 seconds (about 67,600 tokens, gpt-oss:120b) would shrink depending on the machine has not been measured under the same conditions.

What I Haven’t Confirmed in This Article

• How reading speed moves when the context length setting is changed, and whether the gap from last time comes from there
• Whether other models show the same shape. I measured only one, gpt-oss:120b
• Two of the seven steps, about 1,100 and about 2,200, carry a different state label in the measurement record from the other five. Their reading speeds fall in line with the other five, so I treated them the same, following the measurement notes
• Input longer than about 67,600 tokens. I have not measured how far it grows on this machine and model

Summary: what I learned by stepping the input

• When the input became 60 times, the time to the first character grew 169 times. It is not a simple proportion
• Growth beyond proportion is because the reading speed itself falls, from 725 tok/s to 216 tok/s
• At about 67,600 tokens it is 313.70 seconds. Over five minutes, with nothing on screen
• The spread of 5 runs is within 5% of the median at every step. These are not scattered numbers
• Sending the same text again takes 0.38 seconds even at about 35,000 tokens. The heavy part is sending a long text for the first time
• There is a gap of more than two times from last article’s 37 seconds. Which condition is the cause is unconfirmed

If you use it by reading long material, what splits the wait is whether you can keep asking questions without changing where the material sits.

The numbers in this article are from August 2026, in this environment (GMKtec EVO-X2 + Ollama 0.30.10, gpt-oss:120b, context length 131,072). Results change with a different model or settings.

Articles on running AI locally are collected on the page below.

Local AI guide

Hardware used for the tests

The machine used for the measurements.

As an Amazon Associate we earn from qualifying purchases.