My Whole Local AI Workstation, Documented: Ubuntu + RTX 3090 + Ollama + ComfyUI

2026年8月7日

I built a workstation at home dedicated to local AI. Ubuntu 24.04, two NVIDIA GPUs, and a full set of Ollama, ComfyUI and Docker. This article publishes every step needed to reproduce that environment from scratch.

The short version: a used GPU (an RTX 3090, ¥130,000–180,000) and free software are enough to build a local AI setup that owes nothing to commercial AI services. The only recurring cost is electricity. What you get is an environment where a 27B-parameter model (B means billion; bigger tends to mean smarter) runs comfortably.

Every step here is based on a real machine: AMD Ryzen 9 3950X / RTX 3090 + RTX 3060 / Ubuntu 24.04.

Specifications and prices are as of April 2026.

The whole picture

Ollama handles chat (LLM inference), ComfyUI handles image generation, and Docker runs development WordPress instances and the like. Having two GPUs means you can chat and generate images at the same time.

Choosing the OS: why Linux

There are three options for local AI — Windows, macOS and Linux — each with strengths and weaknesses.

Windows is the easiest to start with, but tools like Docker assume WSL2, which adds work. Ollama has a native Windows build, but most of the information in this field targets Linux, and there are reports of troubleshooting taking longer.

macOS has the appeal of unified memory on Apple Silicon (M1 and later), but does not support NVIDIA GPUs. That is fundamentally at odds with a build based around a used GPU.

Linux takes more effort to set up but is the most stable for AI work. Both Ollama and ComfyUI are most stable on Linux, and new features land there first. Docker Engine runs natively, with no virtual machine in between.

That is why I chose Ubuntu 24.04 LTS. LTS (long-term support) guarantees five years of security updates, so once it is built you can leave it alone.

Hardware and what it cost

The big spend is the RTX 3090. The used market runs from ¥130,000 to ¥180,000, so the trick is judging the condition of the individual card.

Choosing a GPU: VRAM decides everything

For local LLMs, the specification that matters most is VRAM capacity. Whether the whole model fits in VRAM changes inference speed dramatically. You can estimate what you need like this:

Required VRAM (GB) = parameters (B) × quantization factor

Rough factors:
  FP16 (full precision) : × 2.0
  Q8  (8-bit quantized) : × 1.1
  Q4  (4-bit quantized) : × 0.6

Example: a 27B model at Q4 = 27 × 0.6 ≈ 16.2 GB → fits comfortably on an RTX 3090 (24GB)
Example: a 70B model at Q4 = 70 × 0.6 ≈ 42 GB → too much for 24GB; needs two cards

Why an RTX 3090? It has 24GB of VRAM, and a 27B model fits with room to spare. An RTX 4090 (also 24GB) is over ¥300,000 new; a used 3090 is ¥130,000–180,000. Same VRAM means the same model sizes fit.

Why add an RTX 3060? 12GB is enough for 8B–9B class models. The idea is to run the heavy work on the 3090 while the 3060 handles side tasks at the same time. At around ¥20,000 used, the value is excellent.

The RTX 3090 draws up to 350W, so leave headroom in the power supply.

Installing Ubuntu 24.04

Nothing unusual here. Boot from a USB stick written with the ISO, format the whole disk as ext4 and install. The default partition layout (using LVM) is fine.

# After installing, bring the system up to date first
sudo apt update && sudo apt upgrade -y

# Install the essentials
sudo apt install -y build-essential git curl wget

NVIDIA driver and CUDA

Installing the NVIDIA driver on Ubuntu 24.04 is far easier than it used to be. No manual repository setup — standard apt commands do it all.

# Check the recommended driver
ubuntu-drivers devices

# Install it automatically
sudo ubuntu-drivers autoinstall

# Reboot
sudo reboot

# After rebooting, check the driver is loaded
nvidia-smi

A successful install gives output like this:

Driver Version: 580.126.09
GPU 0: NVIDIA GeForce RTX 3060   12288 MiB
GPU 1: NVIDIA GeForce RTX 3090   24576 MiB

If both cards show up, you are done. The 580 series was the current stable driver as of April 2026.

About CUDA. Both Ollama and ComfyUI bundle the CUDA runtime, so you normally do not need to install the CUDA Toolkit separately. You only need it if you are developing directly against PyTorch.

Installing Ollama

Ollama runs large language models locally. Normally an LLM inference setup means managing Python, PyTorch and a pile of libraries; Ollama bundles all of that, and installing is a one-liner.

# Install Ollama (official script)
curl -fsSL https://ollama.com/install.sh | sh

# Check the version
ollama --version
# ollama version is 0.20.2

# Try it: pull a small model and run it
ollama run qwen3:8b

That is all it takes to have local AI chat running. The first run spends a few minutes downloading the model (about 5.2GB for qwen3:8b); after that it starts in seconds.

The models installed here

ollama pull qwen3:8b        # 5.2GB - light and fast, everyday use
ollama pull qwen3:14b       # 9.3GB - balanced
ollama pull qwen3.5:9b      # 6.6GB - latest Qwen generation
ollama pull qwen3.5:27b     # 17GB  - high quality, RTX 3090 recommended
ollama pull gemma4          # 9.6GB - from Google, good in Japanese too
ollama pull gemma4:26b      # 17GB  - the large Google model
ollama pull minicpm-v:8b    # 5.5GB - multimodal, understands images

Rough guidance on which to use:

Everyday chat: qwen3:8b or qwen3.5:9b. Comfortable even on the RTX 3060.

Involved questions and code generation: qwen3.5:27b or gemma4:26b. These use the full 24GB of the RTX 3090.

Asking about an image: minicpm-v:8b, a multimodal model that accepts images as input.

Installing ComfyUI

ComfyUI is a node-based image generation tool. Automatic1111’s Web UI is a comparable option, but ComfyUI gives you more freedom over the workflow — rearranging nodes lets you build complex pipelines. That is why I use it here.

# Create a Python virtual environment
sudo apt install -y python3.12-venv
python3 -m venv ~/comfyui-venv
source ~/comfyui-venv/bin/activate

# Clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI.git ~/ComfyUI
cd ~/ComfyUI

# Install PyTorch (CUDA build)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install ComfyUI's dependencies
pip install -r requirements.txt

# Start it
python main.py --listen 0.0.0.0 --port 8188

Open http://localhost:8188 in a browser and the ComfyUI interface appears.

Choosing the GPU. ComfyUI uses GPU 0 by default. If your RTX 3090 is in slot 1, either pass --cuda-device 1 at startup or set the CUDA_VISIBLE_DEVICES environment variable.

# Start it on the RTX 3090 (GPU 1)
CUDA_VISIBLE_DEVICES=1 python main.py --listen 0.0.0.0 --port 8188

Setting up Docker

Docker is here to run development WordPress instances, databases and so on. There are two ways to install it — the OS package (docker.io) or Docker’s official repository — and the official repository is newer and more stable. That is what I use.

# Add Docker's official repository and install
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Add your user to the docker group (so you can run it without sudo)
sudo usermod -aG docker $USER
newgrp docker

# Check the version
docker --version
# Docker version 29.3.1

On this machine Docker runs three WordPress development environments (ports 8080–8082) and phpMyAdmin (8083), all managed with docker compose, so starting and stopping is one command.

Running two GPUs

Notes on the software side of putting an RTX 3090 and an RTX 3060 in the same machine.

One driver install covers both. You do not need separate drivers for GPU 0 and GPU 1. If nvidia-smi shows both, you are fine.

Ollama uses both automatically. No configuration needed. Run a model that does not fit on one card and Ollama splits it across the two.

ComfyUI uses one card, chosen explicitly. Image generation completes on a single GPU, so point it at the one with the most VRAM — the RTX 3090.

# Check what is detected
nvidia-smi -L
# GPU 0: NVIDIA GeForce RTX 3060 (UUID: GPU-xxxx)
# GPU 1: NVIDIA GeForce RTX 3090 (UUID: GPU-xxxx)

# Ollama environment variables (if needed)
# To use one specific GPU:
CUDA_VISIBLE_DEVICES=1 ollama serve
# To use both (the default): set nothing — Ollama finds all GPUs

About PCIe slots. Put the RTX 3090 in a x16 slot. The RTX 3060 works at x16 or x8, but dropping to x4 costs performance through lack of bandwidth. Check your motherboard manual for how many lanes each slot gets when two cards are installed.

About power. An RTX 3090 (up to 350W) plus an RTX 3060 (up to 170W) plus the CPU and everything else can approach 700W at peak. That is why an 850W supply is the recommendation.

Benchmarks

How fast is it in practice? Inference speed by model in Ollama:

The 8B class exceeds 100 tok/s (tokens per second — the chunks of text generated each second; higher is faster), which feels instant. Even the 27B class runs around 25 tok/s, comfortably faster than you can read.

The RTX 3060 runs the same models, but memory bandwidth (360 vs 936 GB/s) holds it to roughly half the speed. Even so, 60 tok/s on an 8B model is perfectly usable.

Monthly electricity

The running cost people ask about is electricity. Estimated monthly cost:

Whole-system idle draw is around 100W. Two GPUs installed still consume very little with no load on them.

During AI chat it runs 170–380W depending on model size — around 170W for an 8B model, around 260W for 27B, and up to around 380W while ComfyUI is generating images.

At about four hours of AI use a day, that is around ¥3,000 a month. The same as ChatGPT Plus (¥3,000/month) or Claude Pro (¥3,000/month) — except you can use it as much as you like, locally. And once the environment is built, the models are yours.

Wrap-up: total cost and living with it

Initial outlay is about ¥256,000. If you already have a PC and are only adding a GPU, the used RTX 3090 (¥130,000–180,000) is the main expense.

Running cost is ¥3,000 a month in electricity. All the software is free.

From year two it costs ¥36,000 a year. For the same money as an annual cloud AI subscription, you get no usage limits, your data stays local, and you can switch freely between models.

As for difficulty: if you can run commands in a terminal, you will be fine. Following these steps takes about half a day.

Hardware used

As an Amazon Associate we earn from qualifying purchases.