Starting Local AI with a Used GPU: The Value of the RTX 30/40 Series
I run Ollama every day on a PC with two GPUs. I bought the RTX 3090 new at launch for about ¥300,000, and picked up the RTX 3060 12GB used for around ¥40,000.
When you go looking for a GPU for local AI, older used cards quickly become candidates. The RTX 3060 12GB in particular gets you 12GB of VRAM for ¥20,000–35,000, which makes it excellent value as an entry card.
That said, the RTX 30 series is a generation where many cards were hammered during the mining boom. You need to understand the used-market caveats before you choose. This article organizes how to pick a used GPU from a local-LLM perspective.
The RTX 30 series and mining
The first thing to know: the RTX 30 series (released 2020–2022) was thrown at cryptocurrency mining in huge numbers.
The RTX 3060 Ti, 3070, 3080, and 3090 especially were bought in bulk for mining, and many of the units now circulating on the used market ran flat out, 24/7, 365 days a year. Fans and VRMs may be worn, so jumping on one just because it’s cheap is risky.
But even within the RTX 30 series, the RTX 3060 12GB is a different story. The 3060 was the first card NVIDIA shipped with LHR (Lite Hash Rate, a mining limiter), holding its hash rate down to roughly 26–36 MH/s. On top of that, its 12GB of VRAM was unnecessary for ETH mining (the DAG was only about 4–5GB), so it felt overpriced for the hash rate it delivered and was unpopular with miners. In other words, a used 3060 12GB is comparatively unlikely to have been thrashed by mining.
The RTX 3060 Ti, by contrast, had a non-LHR version and was hugely popular for mining thanks to its excellent efficiency of about 60 MH/s at 120W. The same goes for the 3070, 3080, and 3090. Used examples of these need careful inspection.
The RTX 40 series (released 2022–2024) came after the mining boom, so abused units are comparatively rare.
The used prices in this article are based on shop prices (with a warranty). Flea-market apps and auctions can be a bit cheaper, but assessing condition is harder, so if this is your first used GPU I recommend buying from a shop.
Why choose a used GPU for local AI = the VRAM value is good
Local-LLM performance is basically decided by the amount of VRAM (for the details, see “I want to run an AI chatbot at home"). Used cards get you the same VRAM for less than new.
Price per GB of VRAM (as of April 2026)
| GPU | VRAM | Price range | Per GB | Notes |
|---|---|---|---|---|
| RTX 3060 12GB (used) | 12GB | ¥20k–35k | ~¥2,300/GB | Cheapest entry point |
| RTX 3080 10GB (used) | 10GB | ¥40k–60k | ~¥5,000/GB | |
| RTX 3080 12GB (used) | 12GB | ¥50k–70k | ~¥5,000/GB | |
| RTX 3090 24GB (used) | 24GB | ¥130k–200k | ~¥6,900/GB | Prices stay high on AI demand |
| RTX 4060 Ti 16GB (used) | 16GB | ¥70k–100k | ~¥5,300/GB | The 16GB version is scarce |
| RTX 4090 24GB (used) | 24GB | ¥300k–380k | ~¥12,500/GB | |
| RTX 5060 Ti 16GB (new) | 16GB | ~¥90k | ~¥5,600/GB | |
| RX 7900 XTX 24GB (new) | 24GB | ~¥180k | ~¥7,500/GB |
Two price bands stand out:
- RTX 3060 12GB (¥20k–35k): the cheapest way to get 12GB. Ideal for getting started with 8B models
- RTX 4060 Ti 16GB (¥70k–100k): a rare option for 16GB of VRAM on the used market. It runs 14B models. Just be careful not to confuse it with the 8GB version (used ¥40k–50k)
The RTX 3090’s 24GB of VRAM is attractive, but AI demand keeps used prices high — ¥130k–200k at shops. It’s hard to call that “a steal." Given that the same budget buys an RX 7900 XTX (new ¥120k–150k / 24GB), if you go for a used 3090 you’ll want to check the warranty and condition carefully.
Used GPU local-LLM ranking
★★★★★ Best value
RTX 3060 12GB (used ¥20k–35k)
If you just want to try it, this is the one. 12GB of VRAM starting in the low ¥20,000s.
| Item | Spec |
|---|---|
| VRAM | 12GB GDDR6 |
| Memory bandwidth | 360 GB/s |
| Runnable models | Qwen 3 8B, Gemma 3 12B (just barely) |
| Generation speed | Qwen 3 8B: ★ 60 tok/s |
| Power draw | 170W |
| Used price | ¥20k–35k (shop) / ¥17k–30k (flea market) |
★ = author-measured (April 2026). The rest are estimates from the estimation formula.
- The memory bandwidth is narrow (360 GB/s), so generation speed trails higher-end GPUs
- 14B models won’t fit. 8B–12B is the realistic ceiling
- It’s an RTX 30-series card, but the LHR limiter plus unneeded 12GB VRAM made it unpopular for mining, so thrashed units are less common. Still, a shop warranty is the safe choice just in case
RTX 4060 Ti 16GB (used ¥70k–100k)
One of the few ways to secure 16GB of VRAM used. That said, the 16GB version is thin on the ground and pricier than the 8GB one. Consider it when you specifically need 16GB.
| Item | Spec |
|---|---|
| VRAM | 16GB GDDR6 |
| Memory bandwidth | 288 GB/s |
| Runnable models | Qwen 3 14B, DeepSeek-R1 14B |
| Generation speed | Qwen 3 14B: 23 tok/s |
| Power draw | 165W |
| Used price | ¥70k–100k (shop; the 16GB version is scarce) |
★ = author-measured (April 2026). The rest are estimates from the estimation formula.
With 16GB, 14B models run. The memory bus is relatively narrow at 128-bit / 288 GB/s, so generation speed trails the RTX 5060 Ti 16GB. You may feel it’s “smart but slow to talk." The low power draw is a plus. And being RTX 40-series, there’s little worry about mining abuse.
★★★★ Recommended
RTX 3080 12GB (used ¥50k–70k)
On top of 12GB of VRAM, it has wide bandwidth (384-bit / 912 GB/s), so generation is faster than the 3060.
| Item | Spec |
|---|---|
| VRAM | 12GB GDDR6X |
| Memory bandwidth | 912 GB/s |
| Runnable models | Qwen 3 8B, Gemma 3 12B |
| Generation speed | Qwen 3 8B: 131 tok/s |
| Power draw | 350W |
| Used price | ¥50k–70k (shop) |
★ = author-measured (April 2026). The rest are estimates from the estimation formula.
Same 12GB as the 3060, but 2.5x the bandwidth. It genuinely feels “quick to reply." The downside is the 350W power draw, so it’s for people with headroom in their PSU. Being a mining-era card, checking the unit’s condition is a must.
RTX 3090 24GB (used ¥130k–200k)
With 24GB of VRAM, it’s top-class for local LLMs. But used prices remain stubbornly high.
| Item | Spec |
|---|---|
| VRAM | 24GB GDDR6X |
| Memory bandwidth | 936 GB/s |
| Runnable models | Qwen 3 32B, Gemma 3 27B, DeepSeek-R1 32B |
| Generation speed | Qwen 3 32B: 32 tok/s |
| Power draw | 350W (high) |
| Used price | ¥130k–200k (shop) / ¥100k–150k (flea market) |
★ = author-measured (April 2026). The rest are estimates from the estimation formula.
- 24GB cards hold their value on AI demand. This isn’t a “cheap used bargain" price band
- At the same price you can buy an RX 7900 XTX (new ¥120k–150k / 24GB). If you want a new-product warranty, consider that too
- At 350W, a 750W+ PSU is recommended
- Ex-mining units are very common. Always check the fan condition and listen for odd noises
- It occupies 3 slots. You need physical room in the case and on the motherboard
- It runs hot. Air conditioning is a must in summer
- You may find one around ¥100k on flea markets, but with no warranty and unknown condition, that’s a risk
RTX 4090 24GB (used ¥300k–380k)
If your budget allows, this is the one. On top of 24GB of VRAM, its bandwidth exceeds 1 TB/s.
| Item | Spec |
|---|---|
| VRAM | 24GB GDDR6X |
| Memory bandwidth | 1,008 GB/s |
| Runnable models | Qwen 3 32B, Gemma 3 27B |
| Generation speed | Qwen 3 32B: 34 tok/s |
| Power draw | 450W |
| Used price | ¥300k–380k (shop) |
★ = author-measured (April 2026). The rest are estimates from the estimation formula.
Same 24GB as the 3090, but 1.5x the generation speed. It’s also top-class for games, VR, and image generation. Being RTX 40-series, there’s little worry about mining abuse. If the price gap with the 3090 has narrowed, the 4090 gives more peace of mind.
★★★ Recommended with caveats
RTX 3080 10GB (used ¥40k–60k)
Cheap, but 10GB is awkward. 8B models have room to spare, but 14B won’t fit. If the price gap with the 12GB 3060 is small, the 3060 is the better VRAM value. The usual mining-era caution applies.
RTX 4080 16GB (used ¥100k–130k)
A fast 16GB GPU. Bandwidth is wide (717 GB/s), but the new RTX 5060 Ti 16GB (¥90k–110k) is cheaper. If a used one costs over ¥100k, buying new is the more rational call.
Not recommended
| GPU | VRAM | Reason |
|---|---|---|
| RTX 3070 8GB | 8GB | Cheap but VRAM-starved. For 8GB, the 3060 12GB has more VRAM |
| RTX 3070 Ti 8GB | 8GB | Same as above |
| RTX 4060 8GB | 8GB | Newer, but 8GB is not enough. A used 4060 Ti 16GB is better |
| RTX 4070 12GB | 12GB | Used ¥60k–80k. Double the price of a 3060 12GB for the same VRAM |
| RTX 4070 Ti 12GB | 12GB | Used ¥70k–90k. Same as above |
Value scatter: used GPUs × local LLMs
How to read this chart: the horizontal axis is used price (¥10k units), the vertical axis is a local-LLM performance score. The closer to the top-left, the better the value.
Things to watch when buying a used GPU
Watch out for ex-mining cards (especially RTX 30-series)
The RTX 30 series was used heavily for cryptocurrency mining. Many units that ran under high load 24/7 are circulating on the used market.
Inspection checkpoints:
- Whether the fans spin smoothly (no odd noises or wobble)
- Check the manufacturing date and serial with GPU-Z (2020–2021 builds have a high mining-use rate)
- How clogged with dust the heatsink is (be wary of uncleaned units)
- A warranty-backed shop (Janpara, Dospara used, Sofmap, etc.) is safer than a private sale on Mercari or the like
- Check the warranty period (varies by shop, but roughly a month)
- “Confirmed working" and “warranty-backed" are not the same thing. Always check whether there’s a warranty
The RTX 40 series came after the mining boom, so this worry is comparatively minor.
Check your power supply
| GPU | TDP | Recommended PSU |
|---|---|---|
| RTX 3060 12GB | 170W | 550W+ |
| RTX 3080 12GB | 350W | 750W+ |
| RTX 3090 24GB | 350W | 750W+ |
| RTX 4060 Ti 16GB | 165W | 550W+ |
| RTX 4090 24GB | 450W | 850W+ |
Running multiple used GPUs: adding VRAM cheaply
For local LLMs, you can pool the VRAM of several GPUs to run a single model. Ollama automatically detects multiple GPUs and splits the load, so no special configuration is needed.
In the used-GPU context, this means you have an option: instead of buying one expensive high-capacity GPU, buy several cheap used GPUs to secure the VRAM.
| Example config | Total VRAM | Runnable models | GPU cost (used) |
|---|---|---|---|
| RTX 3060 12GB × 1 | 12GB | 8B models | ¥20k–35k |
| RTX 3060 12GB × 2 | 24GB | 27B–32B models | ¥40k–70k |
| RTX 3060 12GB × 3 | 36GB | 32B models (with room) | ¥60k–105k |
| RTX 3060 12GB × 4 | 48GB | 70B models (quantized) | ¥80k–140k |
Two RTX 3060 12GBs (24GB total, ¥40k–70k) secure the same 24GB of VRAM for less than half the price of a used 3090 (¥130k–200k). Speed drops versus a single-card setup, but you cross the line of “32B models will run."
One caveat: you can’t mix NVIDIA and AMD GPUs. Their AI compute stacks differ (CUDA vs ROCm), so you have to stick to one GPU vendor. You’ll also need to check your motherboard’s PCIe slot count, the physical size of your case, and PSU capacity (750W+ for two cards, 1000W+ for three or more).
New vs used: which should you choose?
| What you value | Recommendation |
|---|---|
| Start as cheaply as possible | RTX 3060 12GB used (¥20k–35k) |
| Run 14B models cheaply | RTX 4060 Ti 16GB used (¥70k–100k) |
| Warranty and peace of mind | RTX 5060 Ti 16GB new (~¥90k–110k) |
| Run 32B models | RX 7900 XTX new (~¥120k–150k) or RTX 3090 used (¥130k–200k) |
| No compromise on speed or smarts | RTX 4090 24GB used (¥300k–380k) |
| The full spread (AI + VR + gaming) | RTX 5090 32GB new (~¥400k) |
My personal take: the easiest used card to recommend is the RTX 3060 12GB (low ¥20,000s to ¥30,000s). It’s cheap and low-risk as an entry card. If you need 16GB, a used RTX 4060 Ti 16GB (¥70k–100k) is a candidate too, but in the same budget band a new RX 7900 XTX (24GB / from ~¥120k) beats it on both VRAM and warranty.
The RTX 3090’s 24GB is appealing, but even used it runs over ¥130k. Meanwhile the RX 7900 XTX (new, 24GB) has dropped to around ¥120k, getting you the same 24GB new and warranty-backed for less. Between “a used 3090 or a new RX 7900 XTX," on Linux I’d lean toward the RX 7900 XTX — on Linux it’s a genuinely viable option.
The prices and specs in this article are as of April 2026. Used prices move daily. Check the latest prices before buying.
Related
Running a Local AI Chatbot at Home: A Budget-by-Budget Guide