Thirty Minutes on My Own Machine — What Would Renting a GPU Have Cost?

2026年8月11日

I once forced a model with 745 billion parameters onto the machine sitting on my desk. It ran, in the sense that it did produce text. At 0.55 tokens per second. Ask it for 1,000 tokens and you are waiting half an hour.

That is not usable. So what should you do when you want to touch a model of that size?

One option is to rent a GPU by the hour. I had heard of it, but I had never sat down and worked out what it would cost for my own use. I have measured numbers from my own machines, so let me use those.

This is where things stood in July 2026. The speeds are measured here; the prices are from published rate cards. I have not actually rented anything. Prices change, so check the official pages before you sign up.

First: how slow was it on my own hardware?

Here are the numbers I have measured so far, all on my own machines.

The boundary is sharp. Whether the model fits in the memory you have changes the experience completely. While it fits, things are comfortable. The moment it spills over, you are waiting tens of minutes.

What lies beyond that boundary is what I wanted to think about here.

What does renting cost?

There are several hourly services. As a reference point, here are RunPod’s published prices, per hour.

The roughly 2x spread between the cheap and expensive ends comes from where the machine lives. Hardware anyone can list is cheaper; hardware in an operator’s own facility costs more. With the cheap end you have to assume availability will vary.

What caught my eye is the bottom row: 288GB. The 202GB model that made me wait half an hour would sit on a single card.

Working it out per 1,000 tokens

This is the part I wanted. Put my measured speed and the rental price on the same footing.

Laying them side by side changed how I think about it. This is not a story about saving money.

On electricity alone, running it here is overwhelmingly cheaper — $0.01 against $0.10, a factor of ten. If renting still makes sense, then what you are buying is the time. You pay ten cents to turn thirty minutes into fifty seconds.

Once, ten cents feels like nothing. Repeat it fifty times while you experiment and it is five dollars. Depending on how you work, the same number reads very differently.

Where is the line between buying and renting?

One more calculation: buying a used graphics card against renting one indefinitely.

If a used 24GB card is around $670 and renting the same class costs $0.34 an hour, the two meet at roughly 2,000 hours. At eight hours a day that is 250 days; twice a week rather than daily and you are looking at nearly five years.

Things I would have liked to know before renting

Reading the rate cards, these are the parts that looked easy to miss.

• Storage is billed separately. Up to 1TB it is quoted at $0.07 per GB per month. Leave model weights sitting there and they cost money while you are not using them
• Forgetting to stop it is the scary failure mode. Billing is by time, so a machine left running quietly piles up charges
• Download time is billed too. While you pull down a few hundred GB of model, the GPU is still rented
• The cheap end has queues. What you want may not be free when you want it

The third one I can speak to from measurement. Loading that 202GB model from my own local disk took 31 minutes (110MB/s). Over a network the conditions differ again, but it is worth holding on to the idea that before anything runs, there is a wait.

In the end there are three roads

When you want to use a large model, it comes down to three.

I compared API prices in a separate article.

What Does Kimi K3 Actually Cost? Frontier Model Pricing Compared

Kimi K3 is an open-weight model released in July 2026. The weights are distributed, so in theory ...

 https://kimono-oyaji.com/en/kimi-k3-pricing-en/

Wrap-up: renting buys time, not savings

• On your own hardware, whether the model fits in memory decides the experience. It fits: 20 tok/s. It spills: 0.55 tok/s (both measured)
• A GPU carrying 288GB holds, on one card, what took me half an hour
• Per 1,000 tokens: about $0.01 and 30 minutes here, about $0.10 and 50 seconds rented
• Electricity is ten times cheaper at home. What the rental sells is time
• The buy-or-rent line sits around 2,000 hours. Long days most days: buy. A few times a month: rent
• Storage is extra, stopping it is on you, and loading time is billed as well

Before working through it, I had assumed “if it will not run here, just rent." Having done the arithmetic, my sense is that local and cloud are not competitors but different jobs. Daily work here, the occasional big one rented. That is where it settled.

I would like to actually rent one and see how close this arithmetic lands. Whether the assumed speed really shows up is the shakiest part of it.

Services and hardware mentioned

The prices in this article came from here. Signing up is free and you pay for what you use. It is an option for occasionally touching a model that will not fit on your own machine. This is a referral link, so a referral benefit comes to me. It does not raise the price for you.

This one is the “cheaper end" I referred to in the article: a marketplace where anyone can list their own hardware. The price comes down, and availability varies in exchange. This is a referral link, so a referral benefit comes to me. It does not raise the price for you (checked 6 August 2026. I have not actually rented from it).

The hardware on the local side. The measured speeds came from these two machines.

As an Amazon Associate we earn from qualifying purchases.

Sources

Checked 31 July 2026. Prices move. The measured speeds are from my own environment.