How Many Tokens per Second Does gpt-oss-120b Generate on a Mini PC’s Integrated GPU?

October 24, 2026

I ran OpenAI’s large model “gpt-oss-120b” on the “EVO-X2” (a mini PC with the Ryzen AI Max+ 395, 128GB of memory of which the OS sees about 122GiB, and a Radeon 8060S integrated GPU) and measured how fast it produces its answer. The model is published together with its weights (the numerical data that make up the AI’s “brain”). The file alone is about 65GB, so can it run on this mini PC’s integrated GPU?

This reflects the state of things as of October 16, 2026.

Purpose: what this article measures

This article measures two things.

• Generation speed: I repeated the same question and measured how many tokens per second the answer text comes out (the speed of writing, not of reading)
• Which device it ran on: whether the model was loaded onto the integrated GPU, an NVIDIA GPU, or the CPU (checked in the ollama server log)

Some things were not measured. The quality or accuracy of the answers, behavior with long inputs, and use by several people at once are outside the scope of this article. See “Limits: what I did not measure” for details.

My aim is that after reading, you have a rough guide: “gpt-oss-120b fits entirely on the EVO-X2’s integrated GPU and writes at around 35 tokens per second.”

Background: key terms

Since a run of technical terms follows, let me sort them out first.

• Parameters: the tuned numerical values inside an AI’s “brain”. The more there are, the bigger the model is called. “117B” means 117 billion
• Token: the unit an AI uses to process text. It does not match the number of characters; a word or a fragment of a word or character counts as one token
• tok/s (tokens per second): the number of tokens processed per second. Larger is faster, and every speed in this article uses this unit. It refers to the speed of writing the answer (generation)
• MoE (Mixture of Experts): a design with many small specialists (experts) in which only the few that are needed are called for each token. Even if the whole model is large, the amount used in a single calculation stays small
• Quantization: a way to make the file smaller by making the model’s numbers coarser. MXFP4, used here, is a format at roughly the 4-bit level
• VRAM: dedicated memory that serves as the GPU’s work desk. On a machine with an integrated GPU such as the EVO-X2, the GPU borrows part of the main memory (128GB, of which the OS sees about 122GiB)
• ollama: software for running AI locally. I used version 0.32.15
• ROCm: the mechanism for doing AI calculations on AMD GPUs. It corresponds to CUDA on NVIDIA

What is gpt-oss-120b? History and design

Before the measurements, let me sort out what kind of model this is. This section is based on the model card (the documentation) and the configuration file (config.json) on Hugging Face (a site that distributes AI models). The sources are collected at the end of the article.

Two open-weight models from OpenAI

The publisher is OpenAI, and the repository on Hugging Face was created on August 4, 2025. The license is Apache-2.0, and I only confirmed that it says Apache-2.0. The README introduces them as “OpenAI’s open-weight models”. Open-weight means that the numerical data of the “brain” can be downloaded by anyone.

A large and a small model were created side by side on the same day. The larger one is the gpt-oss-120b of this article, and the smaller one is gpt-oss-20b. By the README’s figures, 120b has 117B in total and 5.1B active (the part used at one time), and 20b has 21B in total and 3.6B active. The “120” in the name can be read as referring roughly to the number of parameters, but I could not find any material that states where the name comes from.

Inside: MoE and alternating attention

According to the configuration file, gpt-oss-120b has 36 layers (I have not confirmed how the log’s “37/37” is counted), and each layer has 128 experts, of which 4 are used for each token. This design produces the gap between 117B in total and 5.1B active. It is a model that needs a lot of room to hold the whole thing, but has a small amount of computation per token.

For attention (the mechanism that decides which parts of the text to focus on), layers that look at the whole text and layers that look only at the most recent 128 tokens (sliding window, a method of looking while shifting a window along) alternate. That is 18 layers each out of 36. Layers that look only at the recent part make the memory (KV cache) grow more slowly on long texts. However, I did not find any material this time that confirms that effect with numbers.

The context length (the length of text that can be handled at once) is 131,072 tokens according to the configuration file. The measurements this time did not use inputs that long.

65GB in MXFP4

According to the README, the model’s MoE weights were post-trained in a format called MXFP4, roughly at the 4-bit level, and all evaluations were also done in this format. For that reason, it is described as fitting on a single 80GB GPU. The file I pulled with ollama this time was also MXFP4, and its size was 65GB.

External evaluation: the Artificial Analysis index

As a third-party evaluation, there is the Intelligence Index from Artificial Analysis (a site that compares AI performance). In the graph in the site’s X post (Intelligence Index v4.3.2), the bar for gpt-oss-120b is labeled 12. This value is a number published by Artificial Analysis, not a measurement by this blog. Please check the original post for the exact value.

Method: how I measured

• I did not specify the feature that shows the reasoning process (think) and left it at the default
• The upper limit on generated tokens was 256, and I used the same prompt every time
• The first run includes the time to load the model, so I excluded it and recorded the five runs from the second onward. From these five, I take the median of the middle three. Dropping the two at the ends reduces the effect of scatter
• On the same day, I also measured another set under the same conditions
• Which device the model was loaded onto was checked in the ollama server log (displayed with journalctl)

I set use_mmap to false to stop the method of reading only the needed parts of the model from the file (mmap) and to load everything into memory at the start. I used this setting because large models sometimes fail to load properly.

I do not use the speed of reading the input (input processing) in the article. Because I repeated the same prompt, ollama remembers and reuses the identical beginning, which gives larger values than the real ones.

Results

Generation speed: 35.19 tokens per second

The five values ranged from 34.96 to 35.28 tok/s. The median of the middle three (35.18, 35.19, 35.26) is 35.19 tok/s, with a spread of 0.08 tok/s. At that rate, it takes about 7.3 seconds to finish writing 256 tokens.

Another set on the same day gave 36.49 tok/s (spread 0.06 tok/s). The value moved by about 4% between sets (35.19 to 36.49 tok/s is +3.7%).

Which device it ran on

The server log showed the device used as “ROCm0 Radeon 8060S Graphics” (the integrated GPU). The log said “offloaded 37/37 layers”, so all layers were loaded onto this GPU. The configuration file says 36 layers, and I have not confirmed how the log’s “37/37” is counted. In the memory breakdown, the model itself was 61,223MiB and the KV cache was 4,608MiB. In the ollama ps display, the size was 64.4GB, of which 64.4GB was on the GPU side.

This machine also has an NVIDIA GPU, but it was not selected in the log and was not used during the measurement. It did not run on the CPU alone either.

Discussion: why about 35 tok/s, and how far to trust it

A speed of about 35 tokens per second means finishing a 256-token answer in about 7.3 seconds. However, this is the value for a single question; I did not measure long inputs or several questions at once.

The MoE design may be part of the reason such a large 117B model ran at this speed. Only the 5.1B active part is used for each token, so the amount of data read each time is much smaller than the whole. However, I did not compare it with models of other designs, such as “faster than other designs”, and this explanation is a guess from the numbers in the configuration file.

All layers fitting on the integrated GPU is probably because the 128GB of memory is shared between the main system and the GPU. A 65GB file does not fit on a GPU with a small dedicated VRAM. I guess it loaded in one go because the GPU had enough usable area to spare, but I did not check the upper-limit setting.

I have not confirmed the cause of the roughly 4% movement between sets. Temperature or other processing inside the machine may have had an effect, but that remains a guess. Rather than stating one number flatly, it is safer to read it as being in the 35 to 36 tok/s range.

Limits: what I did not measure

• Answer quality and accuracy: I did not measure the quality of Japanese output, the rate of correct answers, or how often it hallucinates (plausible-sounding mistakes). Speed and how good the answers are are separate matters
• Long inputs: The input-processing figures are unusable because of the cache effect, and I did not measure changes caused by long inputs. I also did not try the 131,072 tokens in the configuration
• Simultaneous use: I ran questions one at a time and did not measure behavior with several at once
• Different reasoning strengths: I did not measure what happens when the reasoning strength (low/medium/high) or the think setting is changed
• Number of measurements: Each set has 5 runs, and the measurement date is a single day, October 16, 2026. Values may move on another day or with another version of ollama
• The Intelligence Index of 12: This is a number published by Artificial Analysis, and this blog has not verified it

Summary: running gpt-oss-120b on the EVO-X2

• gpt-oss-120b (MXFP4, 65GB) was loaded entirely onto the EVO-X2’s integrated GPU (the log shows 37/37) (model 61,223MiB, KV cache 4,608MiB)
• Generation speed was 35.19 tok/s (generating up to 256 tokens, median of the middle three of five runs, spread 0.08 tok/s)
• Another set on the same day gave 36.49 tok/s, a movement of about 4% (+3.7%) between sets
• The NVIDIA GPU was not used, and it was not CPU-only either

If you try it next, why not check the quality of its Japanese replies for your own use? For a guide to speed, please refer to the values in this article.

I will keep following how far large models can be run on machines at hand in this blog’s local LLM articles.

References

• gpt-oss-120b model card (README)
• gpt-oss-120b configuration file
• Repository information (creation date, license): gpt-oss-120b / gpt-oss-20b
• gpt-oss-20b configuration file
• Artificial Analysis X post (Intelligence Index graph)
• The Hugging Face information above was retrieved on October 16, 2026