Trying Ornith-1.0 on Home Hardware — Measuring Speed and How Well It Reasons

2026年8月9日

I wanted to find out how far a code-focused local LLM (large language model — the kind of AI that writes text and code) can be pushed on hardware you would actually have at home. The subject is Ornith-1.0, a fairly new family of models whose pitch is that self-improving reinforcement learning teaches it “how to solve things" rather than just the answers. Built on Gemma 4 and Qwen 3.5, it spans 9B up to 397B. Here I took the two that realistically fit on home hardware — 35B (MoE) and 9B (dense) — and ran them on two machines: a desktop with an RTX 3090, and a mini PC (Ryzen AI Max+ 395, 128GB unified memory — a design where the CPU and GPU share the same memory, so a large slice can be used as GPU memory). I measured speed and how well they reason.

The conclusion first: the 35B (MoE) fits on a 24GB graphics card and runs at around 95 tokens per second, solving both reasoning and code problems reliably. The 9B is light and quick, but it occasionally ran away with itself during generation, so on dependability the 35B was a step ahead. Below, speed first, then reasoning.

The machines

As an Amazon Associate we earn from qualifying purchases.

Speed split cleanly along bandwidth and architecture

Generation speed first. The numbers are tokens generated per second.

On the RTX 3090, the 35B and the 9B ran at almost the same speed. The 35B has far more total parameters yet does not slow down, because MoE (mixture of experts) means only a small part of it is used at any one time. On the mini PC, though, the same 35B managed 55.9 t/s while the smaller 9B came in slower at 32.1 t/s. A smaller model being slower is counter-intuitive, but a dense model reads every layer on every pass, which works against a machine whose unified memory has modest bandwidth (how many gigabytes per second it can move to and from memory). It was a clean example of MoE and dense swapping places on speed.

Reasoning is where the gap showed; code was full marks for both

After speed I counted correct answers on problems with a single right answer: eight reasoning problems including logic traps and multi-step arithmetic, and ten code-comprehension problems such as predicting output and spotting bugs. All of it was run with settings that keep answers from varying between runs (the same input gives the same answer).

Code comprehension was perfect at both sizes. Predicting the output of a list comprehension, the mutable-default-argument trap, off-by-one index bugs — none of the usual stumbling blocks broke them. The “built for code" claim has something behind it. On reasoning the 9B dropped exactly one, and the 35B took full marks. The one it lost was the classic “how many minutes with 100 machines" trap, and the larger model showed a tendency to work carefully through to the end. The difference in ability seems to surface in general reasoning rather than in code.

I also tried real coding tasks

Beyond predicting output and spotting bugs, I checked something closer to real work. Using my own coding-scoring harness, I had each model solve seven tasks covering code generation, refactoring and debugging, then ran the generated code in an isolated environment (a container with networking cut off) to score it. Answers were fixed to be non-varying, and each task was attempted once.

The 35B passed all seven on both machines, with quality landing around 96 on each. That is comfortably enough to lean on for everyday code. The 9B passed everything on the mini PC, but on the RTX 3090 one task went into a long generation loop and timed out. Even with settings that should make answers stable, that kind of runaway appears occasionally, so for work you want to hand over and forget, the 35B was the safer bet. Lightness points to the 9B; dependability points to the 35B.

Fit it all on one card, or keep several resident

What struck me most was how differently the two machines behave. The RTX 3090 takes the whole 35B (21GB) into the GPU and answers at 95 t/s. If you want a single response quickly, a MoE model that fits inside 24GB suits it well.

The mini PC cannot match that speed, but its 128GB of unified memory let me keep Ornith’s 35B and 9B, plus a separate 30B code model, all resident on the GPU at once during measurement. That is over 47GB allocated — a configuration a 24GB card cannot hold. Fast on the 3090, room to keep several around on the mini PC: the split shows up directly in the numbers. If you want to switch between several models locally, spare capacity translates straight into convenience.

Wrap-up

Within what I ran locally, Ornith-1.0’s 35B (MoE) fits on a 24GB graphics card, stays fast, and solves both reasoning and code reliably. The 9B still takes full marks on code and is a perfectly reasonable choice if lightness matters more. Neither speed nor accuracy fell apart at Q4 quantisation, which makes running a code-focused model day to day on home hardware look realistic. Next time I plan to look at tasks closer to actual coding work, including how the waiting time feels in use.

Measurement conditions: ollama / Q4_K_M / median of three runs generating 256 tokens. Reasoning and code were measured with answers fixed to be non-varying. These are measurements from my own environment and vary with model, quantisation and settings.