How Many Tokens per Second Does Qwen3.8 27B Produce on an RTX 3090?

October 20, 2026

Qwen3.8 27B (about 27.8B parameters in total on Hugging Face) has been released on Hugging Face, a site for distributing AI models. The publisher is "Qwen" (the organization name on Hugging Face), and the license is Apache-2.0. The quantized file is about 16.5GB, so it looks like it should fit in the 24GB of VRAM on an RTX 3090.

If you run it on a GPU at home, you will want to know how fast the text appears. So I installed Qwen3.8 27B on an Ubuntu machine with an RTX 3090 and measured the generation speed with ollama. Does the speed change when the input gets longer?

This reflects the state of things as of October 16, 2026.

Purpose: what this article measures

This article measures two things.

• Generation speed: I repeated the same question and measured how many tokens per second the answer text comes out (the writing speed, not the reading speed)
• Change with input length: I varied the question length to 105, 1,190, and 3,469 tokens and looked at how the speed of reading the input and the generation speed move

There are also things I did not measure. The quality and accuracy of the Japanese output, behavior with images, and speed comparisons with other models are outside the scope of this article. See "Limitations: what I did not measure" for details.

The goal is that, after reading, you have a rough guide for running a 27B model on a 3090: "around 40 tokens per second, using about 18 to 19GB of VRAM."

Background: terms explained

Because the technical terms pile up, I will sort them out first.

• Token: the unit in which an AI processes text. It does not match the character count; a word or a fragment of characters counts as one token
• tok/s (tokens per second): the number of tokens processed per second. Larger is faster, and every speed in this article uses this unit
• Input processing and generation: an AI first reads the question (input processing, also called prefill) and then writes the answer one token at a time (generation). The reading speed and the writing speed can be measured separately
• VRAM: the dedicated memory on a GPU. If the whole model fits here, it can run on the GPU alone
• Quantization: a method of making the model’s numbers coarser to shrink the file. The UD-Q4_K_M used here is a format squeezed to roughly 4 bits, and the file is 16.5GB
• ollama: software for running AI locally. I used version 0.30.5
• Dense model: a design that uses the whole model for every question. It is the counterpart of MoE, which uses only the parts it needs. The 27B is dense

What is Qwen3.8 27B? History and design

Before the measurements, I will sort out where this model comes from, because knowing the background of the numbers changes how you read the results. This section is based on official materials I checked, and the sources are collected at the end of the article.

The Qwen lineage and the road to 3.8

The official README of Qwen3.8 describes this generation as "the most capable generation of Qwen’s open model lineup so far, following the spread of Qwen3.5 and 3.6." The Hugging Face repository for the 27B version was created on August 5, 2026. The creation date of a repository does not always match the release date, but it is a rough indication of when it came out.

Going back, the repositories for Qwen3’s 32B and 235B-A22B (an MoE model that uses 22 billion of its 235 billion total parameters at a time) were created on April 27, 2025. Qwen3.5 came in February 2026: the 397B-A17B on February 16, and the 27B and 122B-A10B on February 24. Qwen3.6 came in April 2026: the 35B-A3B on April 15 and the 27B on April 21. In about half a year, the generations advanced through 3.5, 3.6, and 3.8.

According to the official READMEs, the features of each generation are as follows. Qwen3.5 is designed to handle images and text together from the start, with a hybrid structure combining Gated Delta Networks and sparse MoE, and support for 201 languages and dialects. Qwen3.6 was built around agentic use, where the model carries out programming on its own, and Thinking Preservation, which retains the earlier flow of thought. For Qwen3.8 27B, the README lists stronger coding, business tasks, research, and long-running agent operation, along with image and video understanding (up to hour-long videos).

If you want to know more about the Qwen lineage, see Part 3 of my Qwen series on this blog.

Why Did Qwen Reopen Its Weights After Closing the Flagship? Part 3: The History

In 2026, Qwen stopped releasing the weights of its flagship with Qwen3.7. Then in August 2026, w ...

 https://kimono-oyaji.com/en/qwen-history-3-qwen35-38-en/

Inside the design: a 3-to-1 attention structure

According to the configuration file (config.json), Qwen3.8 27B has 64 layers and belongs to the same family as Qwen3.5 (qwen3_5). The 27B models of 3.5, 3.6, and 3.8 had the same layer-structure numbers.

The 64 layers are a block of four layers, three layers that use a mechanism called Gated DeltaNet followed by one layer of ordinary attention (Gated Attention), repeated 16 times. This is called a 3-to-1 hybrid structure. With ordinary attention, the memory (KV cache) grows as the text gets longer. In this design, only 16 of the 64 layers use ordinary attention. The number of KV heads is also small, at 4. It reads like a design that keeps the KV cache small even for long text, but I did not find any material that confirms this effect with numbers this time.

The context length is described as up to 262,144 tokens by default and up to 1,000,000 tokens when extended. The model has also been trained for MTP (Multi-Token Prediction), which predicts several tokens at once, and the configuration file lists one such layer. However, I did not confirm whether MTP was used in this measurement, and I did not measure its effect. A vision encoder (the part that reads images) is also included, but this time I measured with text only.

External evaluation: the Artificial Analysis index

As a third-party evaluation, there is the Intelligence Index from Artificial Analysis (a site that compares AI performance). In the chart in the site’s X post (Intelligence Index v4.3.2), the bar for Qwen3.8 27B is printed with 34. This value is a number published by Artificial Analysis, not a measurement from this blog. Please check the original post for the exact value.

The Qwen3.8 README also contains a comparison table for Terminal Bench 2.1. The 27B scores 73.0, Qwen3.6-27B scores 63.4, and Qwen3.7-Plus scores 64.0, but all of these are values the developer published itself.

Method: how I measured

I used a single machine running Ubuntu. The GPU is an RTX 3090 (24GB of VRAM) and the software is ollama 0.30.5. For the model I used UD-Q4_K_M (16.5GB) from unsloth/Qwen3.8-27B-GGUF on Hugging Face.

Measurement 1: generation speed

• I turned the thinking feature (think) off and fixed temperature (output randomness) at 0 and seed (the random seed) at 1
• The cap on generated tokens was 256, and I used the same prompt every time, asking for a 400-character explanation in Japanese
• I excluded the first run because it includes the time to load the model into VRAM, and recorded the five runs from the second onward. From these five, I take the median of the middle three. Dropping the two at the ends reduces the influence of scatter

Measurement 2: varying the input length

• I used three input lengths, 105, 1,190, and 3,469 tokens, with five runs each and the median of the middle three
• The context length (how much text can be handled at once) was 8192, and the cap on generated tokens was 64
• Each time, I changed the opening sentence of the input. ollama remembers and reuses inputs that start the same way, so I needed to defeat the cache

Discarded values

When I repeated the same input, input processing showed 20,000 to 65,000 tok/s. This was the result of ollama reusing an input with the same beginning, not the speed at which it actually read the input. I discarded these values as invalid and did not use them in any number in the text. In Measurement 1 the input was short, so I do not treat the input processing speed as a reference.

Results

Generation speed was 38.9 tokens per second

The five values ranged from 38.73 to 38.98 tok/s. The median of the middle three (38.79, 38.88, 38.93) is 38.88 tok/s, and the spread is 0.14 tok/s. That works out to about 6.6 seconds to finish writing 256 tokens.

VRAM usage was 19.3GB, and all layers of the model fit on the GPU. The highest temperature during the measurement was 76℃.

Input length and speed

As the input got longer, input processing rose from 556 tok/s to 1,285 tok/s. Generation stayed in the range of 39.75 to 40.64 tok/s, and did not drop much even with longer input. In this measurement, VRAM usage was 17.9GB and the highest temperature was 83℃.

The values in Measurement 2 came out slightly higher than the generation speed in Measurement 1 (38.88 tok/s). Measurement 2 stopped generation at 64 tokens, so the conditions differ. When comparing, please read each value together with its conditions.

Discussion: how to read the speed and VRAM figures

A speed of about 39 to 40 tokens per second works out to about 6.6 seconds for 256 tokens and about 1.6 seconds for 64 tokens. However, this is the value for a single question run alone, and I did not measure what happens when several questions are handled at the same time.

Input processing looks faster the longer the input is, at 556 tok/s for 105 tokens and 1,285 tok/s for 3,469 tokens. I suspect that with a short input, the share of time spent on preparation for reading is large. I did not confirm this reason this time.

The fact that generation speed barely moved with input length may be related to the design with few ordinary-attention layers. However, the longest input I measured was 3,469 tokens. If this hypothesis is correct, speed should hold up even with longer inputs, but I have not confirmed that.

The VRAM usage differs between 19.3GB (Measurement 1) and 17.9GB (Measurement 2), and the different measurement conditions are probably part of the reason. The model file is 16.5GB, so the rest is working memory. Of the 24GB of VRAM, Measurement 1 used 19.3GB, leaving about 4.7GB free.

Limitations: what I did not measure

• Japanese quality and accuracy: I did not measure how well the text reads, the rate of correct answers, or how often it hallucinates. Speed and the quality of the answers are separate matters
• Speed comparison with other models: I did not measure other models or other machines, so I make no claim about which is faster or slower
• Long inputs: I did not measure inputs longer than 3,469 tokens, or use up to 262,144 tokens
• Simultaneous use: I sent questions one at a time and did not measure how it behaves with several at once
• Quantization differences: Only UD-Q4_K_M was tested, and I did not measure the difference from other formats
• MTP, images, and video: Not used (whether MTP was used is unconfirmed)
• Number of measurements: Five runs per condition, on a single measurement day, October 16, 2026. Values can move on another day or with another version of ollama
• The Intelligence Index value of 34: This is a number published by Artificial Analysis and has not been verified on this blog (I did not measure Japanese quality or accuracy)

Summary: results of running a 27B on a 3090

• Qwen3.8 27B in UD-Q4_K_M (16.5GB) fit entirely on the RTX 3090 (24GB of VRAM), using 17.9 to 19.3GB of VRAM
• Generation speed was 38.88 tok/s for a question asking for 400 Japanese characters (generating up to 256 tokens)
• With input from 105 to 3,469 tokens, generation speed was 39.75 to 40.64 tok/s and input processing was 556 to 1,285 tok/s
• When the same input was repeated, input processing showed over 20,000 tok/s, but that was an invalid value caused by cache reuse

If you try it, I suggest checking how well it handles Japanese conversation for your own use. Please use the values in this article as a guide to speed.

I will keep following how far a 27B dense model can go for everyday use in this blog’s local LLM articles.

References

• Qwen3.8-27B model card (README, config.json, chat_template): https://huggingface.co/Qwen/Qwen3.8-27B
• Qwen3.8-27B repository information (creation date, license): https://huggingface.co/api/models/Qwen/Qwen3.8-27B
• Qwen3-32B: https://huggingface.co/api/models/Qwen/Qwen3-32B / Qwen3-235B-A22B: https://huggingface.co/api/models/Qwen/Qwen3-235B-A22B
• Qwen3.5-397B-A17B: https://huggingface.co/api/models/Qwen/Qwen3.5-397B-A17B / Qwen3.5-27B: https://huggingface.co/Qwen/Qwen3.5-27B / Qwen3.5-122B-A10B: https://huggingface.co/api/models/Qwen/Qwen3.5-122B-A10B
• Qwen3.6-35B-A3B: https://huggingface.co/api/models/Qwen/Qwen3.6-35B-A3B / Qwen3.6-27B: https://huggingface.co/Qwen/Qwen3.6-27B
• Quantized file (UD-Q4_K_M, 16.5GB): https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
• Artificial Analysis X post (Intelligence Index chart): https://x.com/artificialanlys/status/2110760441240752399
• The Hugging Face information above was retrieved on October 16, 2026