How Many Tokens per Second Does Hy3 Produce on a Mini PC at 1-Bit Quantization?

October 23, 2026

I ran Tencent’s Hy3 (295B parameters), compressed to an extremely small size, on the EVO-X2 mini PC. The compression is about 1.75 bits per number, roughly one-ninth of the original 16-bit (BF16) precision. This article is a record of speed. I did not measure the quality of the answers, so I do not know it. Quantization to the 1-bit range is generally said to lower quality, but I have not confirmed how this version fares. Running fast and giving useful answers are two separate matters.

How fast can a model of around 300 billion parameters write text on the integrated GPU of a mini PC with 128GB of memory (about 122GiB visible to the OS)?

This reflects the state of things as of October 16, 2026.

Purpose: what this article measured

I measured the following two things.

• Speed of reading the input: how many tokens per second the machine processes when given a 512-token text to read (the reading speed)
• Speed of writing text: how many tokens per second come out when the model writes 128 tokens (the writing speed, which is closer to what you feel in use)

There are also things I did not measure. I did not measure the quality of the answers. Running on ollama, long inputs, several simultaneous questions, and other levels of compression are also outside the scope of this article. Details are in “Limitations: what I did not measure".

My aim is that after reading, you have a rough idea of the speed: a 300B-class model compressed to the 1-bit range fits on a mini PC of this class and writes at around 18 tokens per second. It is not material for judging quality.

Background: terms explained

Many technical terms follow, so let me sort them out first.

• Parameters: the tuned numbers inside an AI. 295B means about 295 billion of them (B stands for a billion)
• MoE (Mixture of Experts): a design with a large number of small “expert" parts, of which only some run for each question. Hy3 has 192 experts and uses 8 of them at a time
• Active parameters: the portion that actually runs when producing one token. For Hy3 it is 21B out of 295B in total
• Tokens and tok/s: a token is the unit in which text is processed, where a word or a fragment of characters counts as one. tok/s is tokens per second, and a larger number is faster
• Input processing (pp512) and generation (tg128): an AI first reads the question (input processing), then writes the answer one token at a time (generation). pp512 is the value measured for processing a 512-token input, and tg128 is the value for generating 128 tokens
• Quantization: a method of making the model’s numbers coarser to shrink the file. IQ1_M, used this time, is an extreme format that shrinks each number to about 1.75 bits on average
• GGUF: a model file format for running models with llama.cpp
• llama.cpp: open-source software for running AI locally
• Vulkan: a common framework for using GPUs, which also works on AMD integrated GPUs
• mmap (memory mapping): a way of loading a file into memory a little at a time. I did not use it this time, and loaded everything into memory first
• MTP (multi-token prediction): a mechanism that guesses the next few tokens ahead to raise the writing speed. I did not use it in this measurement

What Hy3 is: its history and design

Before the measurement, let me sort out where this model comes from. This section is based on what I checked on the official pages of Hugging Face (a site that distributes AI models). The sources are collected at the end of the article.

Publisher and release timing

Hy3 is published by the organization “tencent" on Hugging Face. The model card (the description on the distribution page) says it was developed by the Tencent Hy Team. The repository was created on July 2, 2026, and the license is Apache-2.0. A repository’s creation date does not necessarily match its release date, so please read it as a rough guide to the release timing.

Its predecessor, Hy3 preview

Hy3 has a predecessor called Hy3 preview. Its repository was created on April 13, 2026, and the official text says the weights were released on April 23. Weights are the collection of numbers that make up the inside of an AI. The scale of 295B total parameters, 21B active, and a 3.8B MTP layer is the same for Hy3 preview and Hy3.

The Hy3 preview card describes it as “the first model trained on a rebuilt foundation". The Hy3 card says that after the preview release in late April, the team collected feedback from more than 50 products and increased additional training (post-training) with higher-quality data. Hy3 can be read as the same-scale foundation, polished with additional training. The license also changed: Hy3 preview uses the Tencent Hy Community License Agreement, and Hy3 uses Apache-2.0.

About the name

I could not find an explanation on the official pages of what “Hy" stands for or what the “3" refers to, so I have not confirmed either. The name “hunyuan" appears in the tags and in a contact email address, but I could not find any text explaining a relationship with Hunyuan (混元), so I do not assert one here.

Internal design: 8 of 192 experts

According to the configuration file (config.json), there are 80 layers and 192 experts, and 8 are selected for each token. There is also one shared expert that everyone uses. The attention mechanism (the part that decides where in the text to focus) has 8 KV (key and value) heads out of 64 heads. The context length is 256K (262,144 tokens), and the vocabulary is 120,832.

Even though the total is 295B, only 21B runs to produce one token. So if the whole model can be held in memory, it is natural to think the writing speed is determined by something close to the size of the 21B active part. However, I did not run a measurement to confirm this.

The developer’s intended way to run it is a server with eight GPUs with large memory, such as the H20-3e. I did not find an official procedure for running it on a mini PC.

The GGUF version and how IQ1_M is made

The files I used are Hy3-GGUF, published by the AngelSlim organization on Hugging Face. There are four types: IQ1_M, Q2_K_XL, Q3_K_M, and Q4_K_M. The IQ1_M file is 89,446,312,384 bytes (about 89.4GB, 83.3GiB). Q4_K_M is about 182GB, so IQ1_M is about half its size.

The IQ1_M README describes it as “extreme". It also explains how it is made: the attention computation and the parts close to the output are kept at high precision, while the “routed experts", which take up most of the capacity, are reduced to low bits. For fine-tuning, it says it used an imatrix, which measures importance with sample text. Even so, I did not measure how far the cuts affect the output in this article.

External evaluation: Artificial Analysis index

As a third-party evaluation, there is the Intelligence Index from Artificial Analysis (an AI performance comparison site). In a chart in the site’s X post (Intelligence Index v4.3.2), the bar for Hy3 is labeled 25. This number was published by Artificial Analysis and is not a measurement by this blog. What was evaluated is the normal-precision version, so it does not indicate the quality of the IQ1_M version I used this time.

Method: how I measured

1.75bpw is the average number of bits per number (bits per weight). I measured with llama-bench, which ships with llama.cpp, using -ngl 99 (put all layers on the GPU), -p 512 -n 128 (512 tokens of input processing and 128 tokens of generation), and -r 5 (repeat 5 times). I did not use mmap and loaded the whole file into memory first (-lm none). From the 5 values, I took the middle 3 after dropping the highest and the lowest, and also recorded their spread.

I did not measure on ollama. The results are for llama.cpp only.

Getting the file was a struggle. A single-connection download from Hugging Face was slow, at about 9MB/s, so I switched to 8 parallel range requests, which gave about 50MB/s in total (measured at 16:38 on October 16).

Results

Generation was 18.25 tok/s, with a small spread of 0.02 tok/s, so it was stable across repeats. Input processing was 171.71 tok/s, with a spread of 1.63 tok/s.

Discussion: what the speed does and does not tell us

18.25 tok/s is the writing speed. That works out to text appearing at around 18 tokens per second, so 128 tokens take about 7 seconds. However, this is the value when only one question is run at a time.

A model with 295B in total ran on an integrated-GPU machine with 128GB of memory (about 122GiB visible to the OS), and I think the file shrinking to 89.4GB is a large part of the reason. The normal-precision BF16 is about 600GB (about 298.8 billion parameters times 2 bytes), which does not fit in the memory of the same machine (about 122GiB visible to the OS).

Writing (18.25 tok/s) is about one-tenth the speed of input processing (171.71 tok/s). I guess this is because the needed parts of the data have to be read again for each token written, but I did not confirm the cause this time.

Whether this speed is “usable" is decided only when the quality of the answers matches it. I think compression to the 1-bit range is likely to increase broken text and mistakes. In this article I did not measure the quality of the Japanese, the accuracy, or how well it follows instructions.

Limitations: what I did not measure

• Quality and accuracy: I did not measure the quality of answers, the naturalness of the Japanese, accuracy, or the frequency of hallucinations, so I do not know them. Quantization to the 1-bit range is generally said to lower quality, but I have not confirmed how this version fares
• Running on ollama: not measured. llama.cpp only
• Long inputs: input was 512 tokens and generation was up to 128 tokens. I did not measure with the 256K context length
• Simultaneous use: I ran one question at a time and did not measure several users at once
• Other quantizations: IQ1_M only. I did not measure Q2_K_XL, Q3_K_M, or Q4_K_M
• MTP: I did not use a file with MTP or speculative decoding
• Number of runs and date: 5 runs, on a single day, October 16, 2026. The values will shift if the llama.cpp version changes
• The Intelligence Index of 25: a number Artificial Analysis published for the normal-precision version, which this blog has not verified

Summary: results with IQ1_M

• The IQ1_M version (89.4GB) of Hy3 (a 295B-total, 21B-active MoE) ran on the EVO-X2’s integrated GPU with llama.cpp b11192 + Vulkan
• Input processing (pp512) was 171.71 tok/s and generation (tg128) was 18.25 tok/s (middle 3 of 5 runs)
• I did not measure quality, so I do not know it (quantization to the 1-bit range is generally said to lower quality, but I have not confirmed how this version fares)

If you try it, I suggest first checking the quality of the answers with questions from your own use. For the speed, use the values in this article as a guide.

I will keep following how far large models can be compressed and still used, in this blog’s articles on local LLMs.

References

• Hy3 model card (README and config.json): https://huggingface.co/tencent/Hy3
• Hy3 repository information (creation date, license, parameter count): https://huggingface.co/api/models/tencent/Hy3
• Hy3 preview model card: https://huggingface.co/tencent/Hy3-preview / repository information: https://huggingface.co/api/models/tencent/Hy3-preview
• Quantized files (AngelSlim/Hy3-GGUF): https://huggingface.co/AngelSlim/Hy3-GGUF
• Artificial Analysis X post (Intelligence Index chart): https://x.com/artificialanlys/status/2110760441240752399
• The Hugging Face information above was retrieved on October 16, 2026