My Whole Local AI Workstation, Documented: Ubuntu + RTX 3090 + Ollama + ComfyUI
I built a workstation at home dedicated to local AI. Ubuntu 24.04, two NVIDIA GPUs, and a full set of Ollama, ComfyUI and Docker. This article publishes every step needed to reproduce that environment from scratch.
The short version: a used GPU (an RTX 3090, ¥130,000–180,000) and free software are enough to build a local AI setup that owes nothing to commercial AI services. The only recurring cost is electricity. What you get is an environment where a 27B-parameter model (B means billion; bigger tends to mean smarter) runs comfortably.
Every step here is based on a real machine: AMD Ryzen 9 3950X / RTX 3090 + RTX 3060 / Ubuntu 24.04.
Specifications and prices are as of April 2026.
The whole picture
qwen3.5 9B / 27B
gemma4 / gemma4:26b
minicpm-v:8b
SDXL + LoRA
ControlNet
Python 3.12 venv
MySQL / phpMyAdmin
managed with docker compose
Initial outlay ≈¥256,000
As of April 2026
Ollama handles chat (LLM inference), ComfyUI handles image generation, and Docker runs development WordPress instances and the like. Having two GPUs means you can chat and generate images at the same time.
Choosing the OS: why Linux
There are three options for local AI — Windows, macOS and Linux — each with strengths and weaknesses.
[kimono_heatmap title="Local AI suitability by OS" note="Assessment as of April 2026. ◎ = best, ○ = good, △ = limited, × = unsupported"]
OS | NVIDIA GPU support | Docker fit | Ease of setup | Stability for AI | Cost
Windows | ○ native | △ needs WSL2 | ◎ full GUI | △ some tools assume WSL2 | △ license cost
macOS | × no NVIDIA | △ needs a VM | ◎ full GUI | △ Apple Silicon only | × expensive hardware
Ubuntu | ◎ native | ◎ native | △ mostly CLI | ◎ most stable | ◎ free
[/kimono_heatmap]
Windows is the easiest to start with, but tools like Docker assume WSL2, which adds work. Ollama has a native Windows build, but most of the information in this field targets Linux, and there are reports of troubleshooting taking longer.
macOS has the appeal of unified memory on Apple Silicon (M1 and later), but does not support NVIDIA GPUs. That is fundamentally at odds with a build based around a used GPU.
Linux takes more effort to set up but is the most stable for AI work. Both Ollama and ComfyUI are most stable on Linux, and new features land there first. Docker Engine runs natively, with no virtual machine in between.
That is why I chose Ubuntu 24.04 LTS. LTS (long-term support) guarantees five years of security updates, so once it is built you can leave it alone.
Hardware and what it cost
[kimono_cost_table title="Hardware cost breakdown" note="★ = bought used. Prices are approximate, as of April 2026″]
Part | Specification | Cost (incl. tax)
CPU | AMD Ryzen 9 3950X (16 cores / 32 threads) ★ | ≈¥25,000
Motherboard | X570 chipset (PCIe 4.0) ★ | ≈¥12,000
Memory | DDR4 64GB (32GB × 2) | ≈¥12,000
GPU 1 | NVIDIA RTX 3090 24GB ★ | ≈¥150,000
GPU 2 | NVIDIA RTX 3060 12GB ★ | ≈¥20,000
Storage | NVMe SSD 1.5TB | ≈¥12,000
PSU | 850W 80PLUS Gold | ≈¥15,000
Case | Full tower (airflow-focused) | ≈¥10,000
Total | − | ≈¥256,000
[/kimono_cost_table]
The big spend is the RTX 3090. The used market runs from ¥130,000 to ¥180,000, so the trick is judging the condition of the individual card.
Choosing a GPU: VRAM decides everything
For local LLMs, the specification that matters most is VRAM capacity. Whether the whole model fits in VRAM changes inference speed dramatically. You can estimate what you need like this:
Required VRAM (GB) = parameters (B) × quantization factor Rough factors: FP16 (full precision) : × 2.0 Q8 (8-bit quantized) : × 1.1 Q4 (4-bit quantized) : × 0.6 Example: a 27B model at Q4 = 27 × 0.6 ≈ 16.2 GB → fits comfortably on an RTX 3090 (24GB) Example: a 70B model at Q4 = 70 × 0.6 ≈ 42 GB → too much for 24GB; needs two cards
Why an RTX 3090? It has 24GB of VRAM, and a 27B model fits with room to spare. An RTX 4090 (also 24GB) is over ¥300,000 new; a used 3090 is ¥130,000–180,000. Same VRAM means the same model sizes fit.
Why add an RTX 3060? 12GB is enough for 8B–9B class models. The idea is to run the heavy work on the 3090 while the 3060 handles side tasks at the same time. At around ¥20,000 used, the value is excellent.
The RTX 3090 draws up to 350W, so leave headroom in the power supply.
Installing Ubuntu 24.04
Nothing unusual here. Boot from a USB stick written with the ISO, format the whole disk as ext4 and install. The default partition layout (using LVM) is fine.
# After installing, bring the system up to date first sudo apt update && sudo apt upgrade -y # Install the essentials sudo apt install -y build-essential git curl wget
NVIDIA driver and CUDA
Installing the NVIDIA driver on Ubuntu 24.04 is far easier than it used to be. No manual repository setup — standard apt commands do it all.
# Check the recommended driver ubuntu-drivers devices # Install it automatically sudo ubuntu-drivers autoinstall # Reboot sudo reboot # After rebooting, check the driver is loaded nvidia-smi
A successful install gives output like this:
Driver Version: 580.126.09 GPU 0: NVIDIA GeForce RTX 3060 12288 MiB GPU 1: NVIDIA GeForce RTX 3090 24576 MiB
If both cards show up, you are done. The 580 series was the current stable driver as of April 2026.
About CUDA. Both Ollama and ComfyUI bundle the CUDA runtime, so you normally do not need to install the CUDA Toolkit separately. You only need it if you are developing directly against PyTorch.
Installing Ollama
Ollama runs large language models locally. Normally an LLM inference setup means managing Python, PyTorch and a pile of libraries; Ollama bundles all of that, and installing is a one-liner.
# Install Ollama (official script) curl -fsSL https://ollama.com/install.sh | sh # Check the version ollama --version # ollama version is 0.20.2 # Try it: pull a small model and run it ollama run qwen3:8b
That is all it takes to have local AI chat running. The first run spends a few minutes downloading the model (about 5.2GB for qwen3:8b); after that it starts in seconds.
The models installed here
ollama pull qwen3:8b # 5.2GB - light and fast, everyday use ollama pull qwen3:14b # 9.3GB - balanced ollama pull qwen3.5:9b # 6.6GB - latest Qwen generation ollama pull qwen3.5:27b # 17GB - high quality, RTX 3090 recommended ollama pull gemma4 # 9.6GB - from Google, good in Japanese too ollama pull gemma4:26b # 17GB - the large Google model ollama pull minicpm-v:8b # 5.5GB - multimodal, understands images
Rough guidance on which to use:
Everyday chat: qwen3:8b or qwen3.5:9b. Comfortable even on the RTX 3060.
Involved questions and code generation: qwen3.5:27b or gemma4:26b. These use the full 24GB of the RTX 3090.
Asking about an image: minicpm-v:8b, a multimodal model that accepts images as input.
Installing ComfyUI
ComfyUI is a node-based image generation tool. Automatic1111’s Web UI is a comparable option, but ComfyUI gives you more freedom over the workflow — rearranging nodes lets you build complex pipelines. That is why I use it here.
# Create a Python virtual environment sudo apt install -y python3.12-venv python3 -m venv ~/comfyui-venv source ~/comfyui-venv/bin/activate # Clone ComfyUI git clone https://github.com/comfyanonymous/ComfyUI.git ~/ComfyUI cd ~/ComfyUI # Install PyTorch (CUDA build) pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 # Install ComfyUI's dependencies pip install -r requirements.txt # Start it python main.py --listen 0.0.0.0 --port 8188
Open http://localhost:8188 in a browser and the ComfyUI interface appears.
Choosing the GPU. ComfyUI uses GPU 0 by default. If your RTX 3090 is in slot 1, either pass --cuda-device 1 at startup or set the CUDA_VISIBLE_DEVICES environment variable.
# Start it on the RTX 3090 (GPU 1) CUDA_VISIBLE_DEVICES=1 python main.py --listen 0.0.0.0 --port 8188
Setting up Docker
Docker is here to run development WordPress instances, databases and so on. There are two ways to install it — the OS package (docker.io) or Docker’s official repository — and the official repository is newer and more stable. That is what I use.
# Add Docker's official repository and install sudo apt install -y ca-certificates curl gnupg sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \ https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt update sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin # Add your user to the docker group (so you can run it without sudo) sudo usermod -aG docker $USER newgrp docker # Check the version docker --version # Docker version 29.3.1
On this machine Docker runs three WordPress development environments (ports 8080–8082) and phpMyAdmin (8083), all managed with docker compose, so starting and stopping is one command.
Running two GPUs
Notes on the software side of putting an RTX 3090 and an RTX 3060 in the same machine.
One driver install covers both. You do not need separate drivers for GPU 0 and GPU 1. If nvidia-smi shows both, you are fine.
Ollama uses both automatically. No configuration needed. Run a model that does not fit on one card and Ollama splits it across the two.
ComfyUI uses one card, chosen explicitly. Image generation completes on a single GPU, so point it at the one with the most VRAM — the RTX 3090.
# Check what is detected nvidia-smi -L # GPU 0: NVIDIA GeForce RTX 3060 (UUID: GPU-xxxx) # GPU 1: NVIDIA GeForce RTX 3090 (UUID: GPU-xxxx) # Ollama environment variables (if needed) # To use one specific GPU: CUDA_VISIBLE_DEVICES=1 ollama serve # To use both (the default): set nothing — Ollama finds all GPUs
About PCIe slots. Put the RTX 3090 in a x16 slot. The RTX 3060 works at x16 or x8, but dropping to x4 costs performance through lack of bandwidth. Check your motherboard manual for how many lanes each slot gets when two cards are installed.
About power. An RTX 3090 (up to 350W) plus an RTX 3060 (up to 170W) plus the CPU and everything else can approach 700W at peak. That is why an 850W supply is the recommendation.
Benchmarks
How fast is it in practice? Inference speed by model in Ollama:
[kimono_bar title="Inference speed by model (RTX 3090)" unit="tok/s" color="#4caf50″ highlight="1,2″ note="Measured on the RTX 3090 alone. Ollama v0.20.2, num_predict=256, median of 3 runs"]
gemma4 | 133.0
qwen3:8b | 126.4
qwen3.5:9b | 78.5
qwen3:14b | 52.3
qwen3.5:27b | 25.5
gemma4:26b | 22.8
[/kimono_bar]
The 8B class exceeds 100 tok/s (tokens per second — the chunks of text generated each second; higher is faster), which feels instant. Even the 27B class runs around 25 tok/s, comfortably faster than you can read.
The RTX 3060 runs the same models, but memory bandwidth (360 vs 936 GB/s) holds it to roughly half the speed. Even so, 60 tok/s on an 8B model is perfectly usable.
Monthly electricity
The running cost people ask about is electricity. Estimated monthly cost:
[kimono_bar title="Monthly electricity by usage pattern" unit="JPY" color="#2196f3″ note="Calculated at ¥30/kWh using the power figures in this article. Daily hours are assumed values"]
Idle (on 24h) | 2160
Light use (8B, 4h/day) | 2400
Heavy use (27B, 4h/day) | 2700
Including image generation (6h/day) | 3700
[/kimono_bar]
Whole-system idle draw is around 100W. Two GPUs installed still consume very little with no load on them.
During AI chat it runs 170–380W depending on model size — around 170W for an 8B model, around 260W for 27B, and up to around 380W while ComfyUI is generating images.
At about four hours of AI use a day, that is around ¥3,000 a month. The same as ChatGPT Plus (¥3,000/month) or Claude Pro (¥3,000/month) — except you can use it as much as you like, locally. And once the environment is built, the models are yours.
[kimono_cost_table title="Monthly cost comparison" note="Prices as of April 2026″]
Service | Monthly cost | Limits
ChatGPT Plus | ¥3,000 | usage caps
Claude Pro | ¥3,000 | usage caps
Local AI (this build) | ≈¥3,000 (electricity) | none
[/kimono_cost_table]
Wrap-up: total cost and living with it
[kimono_cost_table title="Total cost" note="Approximate, as of April 2026″]
Item | Cost
Hardware, all in | ≈¥256,000
Software (OS, Ollama, ComfyUI, Docker) | ¥0 (all free)
Electricity per month | ≈¥3,000
First year total | ≈¥292,000
Year two onward (annual) | ≈¥36,000
[/kimono_cost_table]
Initial outlay is about ¥256,000. If you already have a PC and are only adding a GPU, the used RTX 3090 (¥130,000–180,000) is the main expense.
Running cost is ¥3,000 a month in electricity. All the software is free.
From year two it costs ¥36,000 a year. For the same money as an annual cloud AI subscription, you get no usage limits, your data stays local, and you can switch freely between models.
As for difficulty: if you can run commands in a terminal, you will be fine. Following these steps takes about half a day.