How Fast Does Nemotron 3.5 Super VL (120B) Write on the EVO-X2 Mini PC? Measured with llama.cpp v0.7.0

October 25, 2026

“Nemotron 3.5 Super VL 120B-A12B," published by NVIDIA (120 billion parameters in total, 12 billion used at a time), is a large model that can also take images and video as input. I ran its Q4_K_M (a 70.1GB file) with llama.cpp v0.7.0 on the mini PC “EVO-X2" (a mini PC with a Ryzen AI Max+ 395 and 128GB of memory; the integrated GPU is a Radeon 8060S) and measured the speed with text only. Does it load on this mini PC, and how fast does it write?

This reflects the state of things as of October 17, 2026.

Purpose: what this article measured

This article measured the following three things.

• Whether it loads: whether the GGUF of Nemotron 3.5 Super VL loads with the Vulkan backend of llama.cpp v0.7.0
• Input-processing speed (pp512): how many tokens per second it processes when fed a 512-token input (the speed of reading)
• Generation speed (tg128): how many tokens per second come out when it writes 128 tokens of text (the speed of writing)

I also recorded the amount of memory used. I did not measure the behavior with image input, the quality or accuracy of the answers, or the behavior with long inputs. Details are in “Limitations: what I did not measure."

The goal is that, after reading, you have a rule of thumb: “a 70.1GB file fits in a mini PC with 128GB of memory and writes at about 24 tokens per second with text only."

Background: terms explained

Several technical terms follow in a row, so let me sort them out first.

• Token: the unit an AI uses when processing text. It does not match the character count; a word or a fragment of a word counts as one token.
• tok/s (tokens per second): the number of tokens processed per second. Larger is faster, and all speeds in this article use this unit.
• llama.cpp: software for running AI locally. I used v0.7.0 (build 620d222) this time.
• llama-bench: a measurement tool that comes with llama.cpp. It measures the processing of a 512-token input (pp512) and the generation of 128 tokens (tg128).
• Vulkan: one of the mechanisms (APIs) for using a GPU. This time I used the Radeon 8060S through Vulkan.
• UMA (unified memory): a design in which the CPU and GPU share the same memory. The EVO-X2 shows no dedicated VRAM figure, so I look at main memory usage instead.
• Quantization: a method of making a model’s numbers coarser to shrink the file. The Q4_K_M used here is a format shrunk to roughly 4 bits, and the file is 70.1GB.
• MoE (Mixture of Experts): a design that holds many small parts called “experts" inside the model and uses only some of them for each token. Even if the total parameter count is large, the computation per token is only for the active parameters.
• Mamba-2: a type of layer that works differently from a Transformer (the usual attention mechanism). Its memory (KV cache) is said to grow less as the text gets longer.
• mmproj: a separate file that holds the parts for reading images. I did not attach it this time, so images cannot be input.

What Nemotron 3.5 Super VL is: history and design trivia

Before the measurement, let me sort out where this model came from. This part was checked against public information on Hugging Face (a site that distributes AI models), and the sources are collected at the end of the article.

The Nemotron lineage

The publisher is NVIDIA. The Release Date in the README is October 15, 2026. The repository creation date on Hugging Face is October 12 for the BF16 version, three days off from the README’s date. I could not confirm the reason.

Nemotron is the name of a series of NVIDIA models. The origin of the name is not written in the materials I checked, so it is unconfirmed. I also could not confirm the meaning of “3.5" or what the abbreviation of the vision encoder name C-RADIO stands for. The README explains “VL" as Vision-Language.

Following the creation dates on Hugging Face, the series has progressed as follows.

Model	Repository creation date	Note
Nemotron 3 Nano 30B-A3B	December 4, 2025	30B in total, 3B active
Nemotron 3 Super 120B-A12B	March 10, 2026	This blog has an article measuring it with ollama
Nemotron 3 Ultra 550B-A55B	June 3, 2026	The distillation teacher (README)
Nemotron 3.5 Lightning 30B-A3B	August 1, 2026	Came out first in the 3.5 generation
Nemotron 3.5 Super VL 120B-A12B	October 12, 2026	The model in this article
Nemotron 3.5 Bolt 74B-A9B	October 7, 2026 (24GB version)	A derivative that compresses Super VL

On the license, the previous generation, Nemotron 3 Super, used the “NVIDIA Nemotron Open Model License," whereas the 3.5 generation uses OpenMDW License 1.1. The README says it can be used both commercially and non-commercially. I only checked a summary of the license text and did not read the full text.

How to read the name: 120B-A12B

“120B-A12B" means 120B (120 billion) parameters in total, of which 12B (12 billion) are used per token. The share used is about 10%. Even though only 12 billion are used, the weights that must be kept on hand cover all 120 billion.

The internal design: the same skeleton as the earlier Super

The README explains that it combines a structure that alternates Mamba-2, MoE, and Attention layers with a vision encoder called C-RADIO and a connector that links vision and language. Counting in the configuration file (config.json), the language side has 88 layers in total: 40 Mamba-2, 40 MoE, and 8 Attention. There are 512 experts, 22 of them used per token, and 1 shared expert that everyone uses. There is 1 MTP layer (a layer that predicts ahead at the same time).

These numbers were the same as in the configuration file of the previous generation, Nemotron 3 Super. The skeleton of the language side is the same as Super; what changed is the added vision encoder, the context-length setting, and the later-stage training. The weights, however, are different, because the training process is different.

For context length, the maximum position in the configuration file is 1,048,576 tokens (1M). For the earlier Super, the configuration file said 262,144 tokens while the README said 1 million tokens, and the two did not match. This time the configuration and the README agree.

The training process: distillation from Ultra

The README’s training process includes the following statements. The numbers are the developer’s description, and this blog did not verify them.

• The language skeleton was initialized from Nemotron 3 Super Base. After training the MTP head on 100B tokens, 1T (one trillion) tokens of distillation were done from the larger Nemotron 3 Ultra Base.
• It used 100B tokens for continued pretraining on Indic languages (languages of the Indian region), 35B tokens for extending to a 1M context, and 67B tokens for software-engineering mid-training.
• Vision-language training was done on C-RADIO and the connector, followed by two stages of supervised fine-tuning.
• Reinforcement learning was done, along with distillation that merges several specialist teachers into one.
• Finally the MTP head was retrained to raise the acceptance rate of speculative decoding (a method in which a small predictor writes the continuation ahead of time and the main model checks it, making generation faster).

Distillation is a method of raising a smaller model by using the outputs of a large model as the example. Ultra is 550B, about 4.6 times the size of Super VL. The meaning of “GA" in the column headings of the README’s evaluation table is not explained, so I could not confirm it.

The distribution formats include BF16 (about 248.9GB) and NVFP4 (the official 4-bit floating point; 75.0GB), among others. The GGUF used this time is the Q4_K_M from lmstudio-community.

The performance NVIDIA shows, and AA’s value

The README’s Benchmarks table carries NVIDIA’s own evaluation. The values in this table are for the BF16 version, not for the Q4_K_M version measured this time. I extract Super VL’s values and some of the values for other models listed alongside it.

Metric	Super VL	Qwen3.5-122B-A10B	Qwen3.8-Flash-Next	GLM-5.3-Flash
GPQA-Diamond	89.52	86.7	92.7	91.2
Terminal-Bench 2.1	60.67	48.70	85.80	82.70
GraphWalks	89.71	86.1	76.1	88.4

The values in the table are NVIDIA’s own evaluations, not third-party verification. There are rows where Super VL is above the other models and rows where it is below. On GraphWalks, Super VL is the highest, while on GPQA-Diamond and Terminal-Bench 2.1, Qwen3.8-Flash-Next and GLM-5.3-Flash are higher.

On the Intelligence Index from Artificial Analysis (AA, a site that compares AI models from a third-party standpoint), Super VL scored 26 (a value printed in an AA article; Intelligence Index v4.3.2). In the graph in an AA X post, the bar for the previous generation, Nemotron 3 Super, was printed as 13 (I could not retrieve the body of the post). AA writes “twice the intelligence, with about 60% fewer output tokens per task." Super VL’s output is about 46k (46 thousand) tokens per task. All of these are numbers published by AA, not measurements by this blog.

Method: how I measured

The machine and software I measured

The machine is the EVO-X2. It is a mini PC with a Ryzen AI Max+ 395 and 128GB of memory. I used the integrated Radeon 8060S GPU through Vulkan. The environment variable GGML_VK_VISIBLE_DEVICES makes only the 8060S visible. The software is llama.cpp v0.7.0 (build 620d222).

Measurement conditions

Item	Details
Model	Nemotron 3.5 Super VL 120B-A12B by lmstudio-community, Q4_K_M (2 split files, 70,095,370,240 bytes in total = 70.1GB)
Parts for images	No mmproj (text only)
Command	llama-bench -ngl 99 -p 512 -n 128 -r 5 -lm none
-ngl 99	A setting that places all layers on the GPU
-lm none	A setting that loads the file into memory up front, instead of the method (mmap) that reads only the parts it needs
Runs	5
Aggregation	Sort the 5 runs from smallest to largest and use the median of the middle 3

The measurement date is October 17, 2026.

Why I do not put ollama numbers in the same table

ollama is separate software from llama.cpp, with a different loading implementation and a different way of measuring. The numbers this time are llama-bench values, so I do not compare them in the same table with the numbers from this blog’s articles measured with ollama.

Results

Whether it loaded

With v0.7.0, it loaded and could be measured. Because I did not attach the image parts, all I could confirm is text input and output.

Speed

Run	Input processing pp512 [tok/s]	Generation tg128 [tok/s]
1	301.45	24.24
2	304.07	24.24
3	314.51	24.31
4	327.49	24.33
5	329.87	24.37
Median of the middle 3	314.51 (spread 23.42)	24.31 (spread 0.09)

For input processing, the 5 runs were scattered from 301.45 to 329.87 tok/s. The spread of the middle three points (304.07, 314.51, and 327.49 tok/s) is wide at 23.42 tok/s, so I treat it as a reference value. The median is 314.51 tok/s, and at that rate a 512-token input takes about 1.6 seconds to read.

Generation ranged from 24.24 to 24.37 tok/s and was nearly consistent. The median is 24.31 tok/s, with a spread of 0.09 tok/s. At that rate, 128 tokens take about 5.3 seconds to write. The values in the table are sorted from smallest to largest, not in the order they were measured.

Memory

The memory used was about 70GB. Because the EVO-X2 shares memory between the CPU and GPU, it shows no dedicated VRAM figure. An amount close to the file size of 70.1GB was placed on the main memory side.

I did not measure temperature or power consumption.

Discussion: what the results suggest

The generation speed of about 24 tokens per second is a value from one condition measured 5 times. I did not measure it with long inputs or with several questions used at once.

Input processing scattered widely across the 5 runs, while generation hardly scattered at all. I could not confirm why input processing scattered. Please read the 314.51 tok/s for input processing as a rough guide that includes run-to-run differences.

The fact about placement is that the 70.1GB file ran on a mini PC with 128GB of memory. Memory usage was about 70GB, within the range of 128GB. The previous generation, Nemotron 3 Super, was measured in another article on the EVO-X2 in a configuration that used NVIDIA GPUs (an RTX 5060 Ti and an RTX 3060) together with main memory. Because the software, quantization, and configuration all differ, I do not line up the speed numbers for comparison. Another example of running a large model on the same mini PC is in the gpt-oss-120b article.

Here an explanation comes to mind: “only 12 billion are used at a time, so even a 120-billion model gives 24 tokens per second." The share used is about 10% (12B ÷ 120B), so fewer weights are read per token. This is a guess, and I did not verify it.

In the performance table, Super VL is high on the item that handles long context (GraphWalks), while rows where other models are above it stand out on coding and agent items. This is my reading of NVIDIA’s own evaluation table, and this measurement did not look at quality.

Limitations: what I did not measure

• Answer quality and accuracy: I did not measure how well the text is written, the rate of correct answers, or how often it hallucinates. Quality is unmeasured. Speed and the quality of answers are separate matters.
• Image and video input: this model is said to accept images and video too, but I ran it with text only, without attaching mmproj. Image input is untested.
• Values for a single question only: these are values from one llama-bench condition (512 input tokens, 128 generated tokens) measured 5 times. I did not measure behavior with long inputs, long generation, or several questions used at once. Behavior at a 1M (one million) token context is also unconfirmed.
• Input-processing values: the spread across the 5 runs is wide at 23.42 tok/s, so they are reference values.
• Switching thinking mode, and MTP: I did not try switching thinking mode (off, medium, on). I did not use MTP or speculative decoding.
• Temperature and power consumption: not measured.
• Comparison with ollama numbers: ollama is separate software. I did not compare against numbers measured with ollama in the same table.
• Differences in quantization: only Q4_K_M. I did not measure the differences from other quantized versions (NVFP4 and so on).
• Day of measurement: a single day, October 17, 2026, this one machine, and one version of v0.7.0 only. On another version or another day, the values may move.
• Performance figures: the evaluation values in the table are NVIDIA’s own evaluations. The Intelligence Index of 26 is a value published by Artificial Analysis (v4.3.2), and this blog did not verify it.

Summary: results of running Nemotron 3.5 Super VL on the EVO-X2

• The Q4_K_M (70.1GB) of Nemotron 3.5 Super VL (120B total, 12B active) ran with llama.cpp v0.7.0 on the EVO-X2, with the Radeon 8060S specified, using text only.
• Input processing was 314.51 tok/s (pp512, median of the middle 3; the spread across the 5 runs is wide at 23.42 tok/s, so a reference value) and generation was 24.31 tok/s (tg128, same).
• The memory used was about 70GB, within the range of 128GB.
• The language-side skeleton has the same configuration as the previous Nemotron 3 Super; the differences are the added vision encoder and the later-stage training.

If you want to try it, first update llama.cpp to v0.7.0 or later and measure the values on your own machine with llama-bench. The values here are only a record of a single run on one machine.

This article did not measure quality or image input. I will keep following them in this blog’s local LLM articles.

References

• Nemotron 3.5 Super VL (README, config.json): https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
• List of Nemotron models (creation dates): https://huggingface.co/api/models?author=nvidia&search=Nemotron-3
• Nemotron 3 Super: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
• GGUF by lmstudio-community: https://huggingface.co/lmstudio-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF
• OpenMDW License 1.1: https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1
• Artificial Analysis article (Nemotron 3.5 Super and Bolt): https://artificialanalysis.ai/articles/nemotron-3-5-super-and-bolt
• The information above was checked on October 17, 2026.