Every team that starts training or serving models hits the same fork: rent GPU time by the hour, or buy the hardware. The honest answer is not a preference. It is a single number — how many hours per month you will actually keep the GPUs busy — and the arithmetic below tells you which side of the line you are on.
The short version
Renting wins when your usage is spiky, exploratory, or below roughly a third of the month. Owning wins when a machine would run most of the time, and the gap widens fast after that. Everything else in this article is how to work out which one you are.
The break-even formula
Compare the hourly cost of each option over the same period.
Renting is simple and honest — you pay for what you use:
Rent_month = hourly_rate × GPUs × hours_used
Owning has to include everything, not just the invoice for the box:
Own_month = hardware/amortisation_months
+ power_kW × 24 × 30 × electricity_rate
+ colocation_or_facility
+ operations_time
Break-even is where the two are equal. Rearranged, the useful form is:
break_even_hours = Own_month / (hourly_rate × GPUs)
Divide that by 720 (hours in a month) and you get the utilisation you must sustain for owning to pay off. Below it, rent. Above it, buy.
The costs everyone forgets
The formula is easy. Getting honest inputs is where estimates fall apart.
Amortisation period, not purchase price. A GPU server is not a one-month cost. Pick the number of months over which you genuinely expect to use it — for current-generation cards, three years is a defensible planning horizon, and residual value at the end is real but should not be counted on.
Power is not the sticker TDP. A GPU rarely sits at 100% draw, but a server also has CPUs, fans, and power-supply losses. Budget on measured wall draw under a realistic load, not on the sum of component TDPs, and remember that in colocation you often pay for the circuit you reserve rather than the watts you consume.
Cooling and space. In colocation this is inside the rack fee. In your own office it is not free — it is an air-conditioning bill and, at real density, a room that was never designed for it.
Operations time. Somebody patches drivers, watches temperatures, replaces a failed PSU. If nobody is assigned to it, the cost has not disappeared; it has become an outage waiting for a bad week.
The idle cost of owning. This is the one that decides the argument. A rented GPU costs nothing when you stop the job. An owned GPU costs the same whether it is training or sitting dark. That asymmetry is the entire reason low-utilisation teams should rent.
When renting is clearly right
- You do not know your workload yet. Requirements change fast in the first months. Renting keeps you free to change your mind about VRAM, card generation, or architecture without owning a decision.
- Your demand is spiky. A model trained once a quarter, inference that peaks for a campaign, a research burst — paying only for the busy hours is simply cheaper.
- You need a card you would not buy. Renting is the sane way to test whether a bigger card actually helps before committing capital to it.
- You need capacity tomorrow. Hardware has lead times; a rented instance does not.
When owning is clearly right
- The machines would run most of the time. Once utilisation is high, the hourly economics stop being close.
- Data cannot leave your control. Regulated work, client data under contract, anything where the answer to "where does this run" has to be a specific building.
- You need predictable latency and no noisy neighbours. A dedicated machine behaves the same on Monday and on Friday.
- You can monetise the idle time. This changes the calculation entirely — see below.
The case that breaks the tie: renting out your own idle capacity
If a machine you own can be listed on a GPU marketplace when you are not using it, the idle hours stop being a pure cost. The effective monthly cost becomes:
Effective_own = Own_month − (rental_rate × GPUs × idle_hours_rented × (1 − platform_fee))
Two honest caveats, because this is where most projections turn into fiction:
- Nobody can guarantee occupancy in advance. How many idle hours actually get rented depends on demand for your card model, your reliability score, and your price relative to everyone else. Treat any fixed promise of utilisation as a red flag.
- The machine has to earn its listing. Marketplaces measure the host and publish the result. A node with poor reliability or a starved CPU sits unrented no matter how good the GPUs are. We documented what that measurement actually looks like in our Vast.ai verification case study — the server there was verified at 98.79% reliability and listed at $1.05 per GPU-hour.
Run your own numbers
We built the GPU ROI calculator around exactly this arithmetic, with live market rates pulled from Vast.ai so the hourly figure is not something we invented. Put in your own hardware cost, electricity rate and expected utilisation, and it returns payback period and annual ROI.
If the answer comes out in favour of owning, the next question is what to actually build — configure a server and we come back with a specification and a price within 24 hours. If it comes out in favour of renting, that is a perfectly good answer and we will tell you so.
A note on the middle path
Buying and renting are not the only two options, and the best answer is often both: own a baseline machine sized for your steady workload, and rent the peaks. You get the economics of ownership where utilisation is high and the flexibility of rental where it is not — without paying for capacity that sits dark eleven months of the year.