Can the AI That Writes No Text, Jev, Run on an RTX 3090? I Measured How Long One Decision Takes on the Local Version of the Same Kind of Model

2026年10月16日

16 October 2026

Some AI models do not write text at all. You hand them a situation and a list of choices, and they pick one answer from the list. TypeSafe announced one of them, Jev, on 15 September 2026 as a “decision model". According to the company, Jev is offered through an API, and no release of its weights is mentioned. A local model of the same kind, APUS-OpenJev, has been published on Hugging Face. It accepts two to 16 choices.

In this article I ran that local model on a desktop PC with an RTX 3090 and measured how long one decision takes and whether the probability of each choice can be read out. Is an AI that writes no text of any use on the PC in front of you?

This is as of October 2026.

What is a decision model such as Jev?

An AI like ChatGPT generates its answer one word (token) at a time. A decision model generates no text. It returns the probability that each of a set of predefined choices (up to 255 for Jev itself) is the right one. One example is sorting an inquiry email into “refund", “pass to a person" or “ask a follow-up question".

According to TypeSafe, input costs $0.042 per million tokens and the response takes 70 to 500 milliseconds. These are the company’s own figures, and I have not checked them.

What I ran is not Jev itself. APUS-OpenJev-v1-9B handles the same kind of question as Jev but is a separate model (Apache-2.0, trained on top of Qwen3.5-9B). I do not know whether it matches Jev in accuracy or speed.

What I measured

On the same desktop PC, I had the model decide five kinds of question, seven times each: three choices, eight choices, 16 choices, yes/no, and a long text (about 1,600 tokens). I measured the time for one decision [ms] and whether the probability of the chosen option could be read out every time.

Machine used for the measurements

Item	Detail
GPU	RTX 3090 (24GB VRAM)
OS	Ubuntu
Runtime	Ollama 0.30.5
Model	APUS-OpenJev-v1-9B (Q4_K_M, 5.2GiB)

How I measured

I built the questions myself in plain Python, following the input format the model card describes. The format writes the state after “Shared state:", lists the choices as JSON, and ends with “Answer:". I ran no external programs. The question used a customer-support case: a customer wrote that the order was two days late and the item was damaged, but that a small discount would be acceptable. The choices were things like “close the ticket", “pass to a person" and “refund".

I had the model output exactly one token at temperature 0, and read the probabilities of the choice labels from the top 20 candidates for that token (logprobs). I renormalized the probabilities so that the labels sum to 1.

Ollama reuses its work on the beginning of an input when it receives the same beginning again (a prefix cache). So I measured two cases separately: changing the input every time, and sending exactly the same input every time. In the changing case I added a different string to the start of each run.

Each condition was run seven times, and I used the median of five runs after dropping the fastest and slowest. The time is the elapsed time from sending the request in Python to receiving the reply. I measured two ways of calling: building the model’s input format myself (generate) and letting Ollama do it (chat). The tables show the generate values.

The Ollama used here is 0.30.5, which has no call dedicated to decision models. So I had the model output one token through the text-generation call (generate) and read the candidate probabilities (logprobs). On 28 September 2026, Ollama 0.35.0 added a dedicated /v1/systemone endpoint (supported models were Nimble and Tev1; 0.35.1 on 29 September added Clef and Clef Flash). I have not checked whether the model in this article works through that endpoint.

Sources: Ollama v0.35.0, v0.35.1, Ollama docs (systemone)

How long does one decision take?

When I changed the input every time, longer inputs took longer. When I sent the same input repeatedly, it took about 0.45 seconds regardless of length.

Question	Input length [tokens]	Input changed each time [ms]	Same input repeated [ms]
3 choices	164	554.4	446.3
8 choices	273	586.8	450.0
16 choices	440	626.7	447.1
Yes/no	136	556.1	448.8
Long text, 3 choices	1,587	980.5	449.2

The spread over seven runs was small. For example, three choices with the input changed each time ranged from 547.0 to 565.6 milliseconds. Calling through chat took 512.1 to 937.3 milliseconds with the input changed and 398.1 to 407.3 milliseconds with the same input, about 40 to 50 milliseconds shorter than generate.

Only the first call, which loads the model, took 4,714 milliseconds. All values in the tables are for an already loaded model.

Can the probability of each choice be read out?

In every condition and every run, all choice labels were in the top 20 candidates, so I could compute the probabilities. The probability of the chosen option differed by question.

Question	Probability of the chosen option (min to max over seven runs, input changed each time)
3 choices	0.816 to 0.903
8 choices	0.968 to 0.979
16 choices	0.895 to 0.924
Yes/no	0.996 to 0.997
Long text, 3 choices	0.906 to 0.948

When I repeated the same input, the probabilities were identical in all seven runs (0.914 for three choices). The chosen option was the same label in every condition and every run.

Where does the floor of about 0.4 seconds come from?

With the same input repeated, it took about 0.45 seconds for both 111 tokens and 1,562 tokens, almost the same. Even when the prefix cache skipped the work on the input and only one answer token remained, about 0.4 seconds were left.

This floor may come from a fixed cost per call rather than from GPU computation. If so, switching to a smaller model should barely lower it. I did not measure a smaller model, so I have not separated the two.

One way to separate them is to measure a smaller decision model (for example, one with a few hundred million parameters) the same way and see whether the floor drops.

What I have not confirmed in this article

• Accuracy. I did not measure whether the local model’s decisions are correct. The 68/80 (85.00%) on the model card is the result on 80 development questions, and the model card itself notes that it is not an independent blind evaluation.
• How reliable the probabilities are. I have not checked whether a decision shown as 0.9 is right nine times out of ten.
• Japanese questions. All the questions here were in English.
• Jev itself. It is API-only, so I did not run it.
• Other GPUs, other models, or sending several decisions at once.

For a third-party evaluation, Red Hat published a comparison on 2 October 2026 (updated 5 October). It reports that Jev does not always beat classifiers, LLM judges or other public decision models in speed and accuracy: Jev ranked first in content safety at 86.20%, while Qwen3.6-35B was higher in prompt injection at 89.31%. The accuracy of the local model is a separate matter from this.

Latest developments (as of 11 October 2026)

These are developments after I wrote the article, listed as facts only. I have not run any of the following.

• 2 October 2026: the llama.cpp server also gained /v1/systemone for decision models. Official GGUFs exist for Julia-1 (144M), Laya (421M), Kev-4B, lev (4B), OpenJev (27B) and Clef, and the median on an RTX PRO 6000 is published as 3 to 43 milliseconds per question. The OpenJev 27B here comes from a different provider (CC BY-NC 4.0, non-commercial only) and is a different model from APUS-OpenJev-v1-9B (Apache-2.0) in this article. The model in this article is not on that list. Sources: Hugging Face Blog (ggml-org), ggml-org decision-models
• 6 October 2026: OpenAI released the Decisions API (gpt-6-luna) in beta. It returns an answer of a fixed form from text and images. Source: OpenAI API changelog
• 7 October 2026: Liquid AI released the weights of the decision model d1-3B (3.12B) and the experimental d1-omni-600M (587M; text and images, or text and audio) under the LFM Open License 1.0. For d1-3B the company reports 8 milliseconds per question on an RTX 4090, a GGUF (Q4_K_M, 1.67GB) is distributed, and it is said to run on llama.cpp’s /v1/systemone. No speed is published for d1-omni-600M. Sources: Liquid AI (Hugging Face Blog), d1-3B, d1-3B-GGUF
• In the Red Hat comparison, the median for Jev (API) is 348.1 milliseconds, of which about 56 milliseconds is said to be the transatlantic round trip. It cannot be compared directly with the roughly 450 milliseconds of the local model here, because the model and the questions differ. Source: Red Hat Developer

Summary: a decision model runs on an RTX 3090, and one decision takes about 0.4 to 1 second

APUS-OpenJev-v1-9B (Q4_K_M) ran on the RTX 3090 desktop PC, and the probability of each choice could be read out every time. One decision took 0.55 to 0.98 seconds when the input changed each time and about 0.45 seconds when the same input was repeated. These values come from this PC and this way of calling, and do not show the speed of Jev itself.

Seeing whether sorting work by having a model return probabilities, instead of writing text, holds up with a local AI was a useful result for anyone using local LLMs seriously. Please also read my other local LLM measurements.

Should you try a decision model?

If you are	Guide
Automating a sort with fixed choices (classifying inquiries, routing tasks)	Worth trying. At 0.4 to 1 second per decision, it is usable one item at a time
Processing thousands of items quickly	I did not measure sending several at once, so measure with your own volume first
In a case where a correct decision matters (such as a safety check)	Accuracy is unmeasured. Check the accuracy on your own data before deciding
Wanting the model to explain in text	Not suitable. A decision model does not return reasons as text

If you are unsure, the surest start is to prepare about 20 examples of your own sorting task and compare the probabilities with the correct answers on the same model.

Sources

• Benchmarking AI decision models against traditional guardrails (Red Hat Developer, 2 October 2026)
• Jev (TypeSafe AI)