The number that travels to finance
Most infrastructure pitches die in the same place: the gap between an engineering benefit and a finance decision. Lower latency, better packing, smoother scaling, all real, none of them a line a CFO can find on a statement. The utilization argument is different, and that difference is the whole point of this piece. It converts cleanly into the three numbers finance already owns: the cost of the asset, the return on it, and the capital you avoid spending next.
Start with the asset cost, because it sets the stakes. Most teams running AI at scale do not own the hardware; they rent it, and the bill is cloud spend, metered by the GPU-hour and landing on the operating statement every month. Rented, an H100 runs roughly $2 to $3.50 per GPU-hour on specialized providers and far higher on hyperscalers, with public on-demand rates reaching about $6.88 on AWS and $12.29 on Azure per H100.²³ Owned, the same card is a $25,000 to $40,000 capital asset, and a full eight-GPU DGX system exceeds $350,000.¹ Either way it is among the most expensive compute a company can buy per unit. The difference is only which statement it hits: a monthly cloud bill or a depreciating asset. That is exactly why the return on it, owned or rented, is the number that matters.
What 5 percent utilization actually costs
Direct measurement across tens of thousands of production clusters puts average GPU utilization near 5 percent, which means organizations are paying for roughly twenty times the capacity they actually use.⁴⁵ Read that as a finance statement, not an engineering one. The buyer is renting, or has bought, powered, and is depreciating, an asset that delivers one-twentieth of its possible work.
Put the unit economics on it, starting with the rented case, because that is where most of the spend lives. Take a blended $3 per GPU-hour, near the low end of cloud rates. A single rented accelerator running continuously is roughly $26,000 a year in cloud spend. At 5 percent utilization, about $1,300 of that bill buys useful work and roughly $24,700 buys idle, billed by the hour, every hour, whether the GPU computes or sits. Scale that to even a modest fleet of a thousand rented accelerators and the idle line alone is on the order of $25 million a year of recurring cloud spend, charged for work that never happened. These are illustrative figures, not a quote, but the order of magnitude is the point: the waste is not a rounding error inside the cloud bill. It is the majority of the cloud bill.
That number moves two ways, and both reach finance. First, recapture: utilization recovered on capacity the customer is already paying for turns idle cloud spend into completed work, so the same monthly bill produces more output, or the same output costs less. The line item a CFO already watches goes further. Second, avoidance: once existing capacity does more, the next commit shrinks or slips. Reserved-instance renewals, capacity expansions, and net-new GPU contracts get smaller or later, because the work is being absorbed by capacity already on contract. Recaptured spend shows up this quarter; avoided spend shows up in the commit the company does not have to sign. Owned fleets get the mirror image of the same gain, which is the next section.
This is why utilization is the rare infrastructure metric that is, in the literal sense, CFO-legible. Every point recovered is direct return on the most expensive line in the compute budget, measured in the company's own dollars, on capacity it is already paying for. There is no other infrastructure lever where a percentage-point improvement maps that directly to recaptured cash and avoided commitment at once.
The owned-side ledger: the depreciation clock
For owned fleets the waste does not disappear, it changes ledgers. Instead of a metered cloud bill it becomes depreciation, and it is the part of the conversation that usually gets missed. The idle GPU is not just failing to earn. It is losing value the entire time it sits there, on a clock the company does not control.
The accounting is under open dispute precisely because the stakes are enormous. Hyperscalers depreciate AI hardware over four-to-six-year useful lives, but NVIDIA now ships a new architecture every twelve to eighteen months, and the real economic life may be closer to two or three years.⁶⁷ Michael Burry's public estimate is that the mismatch could understate industry depreciation by roughly $176 billion across 2026 to 2028, with reported operating income at some operators more than 20 percent above economic reality.⁸ Microsoft's CEO framed the same tension plainly, saying he did not want to be stuck with four or five years of depreciation on a single generation.⁹ The hyperscalers' own divergence proves the variable is live: in early 2025 Amazon shortened a subset of server lives while Meta extended its estimate, under identical technology.⁶¹⁰
The implication for utilization is direct and underappreciated. Depreciation runs on calendar time, not on usage. A GPU bought today loses a meaningful fraction of its economic value over the next two to three years whether it runs at 90 percent or 5 percent. So idle time is not merely forgone earnings, it is forgone earnings on a depreciating asset whose window to earn is short and shrinking. Every quarter a GPU sits underused is a quarter of its abbreviated economic life spent producing nothing while the asset value bleeds off the balance sheet. Recovered utilization is the only thing that converts that bleed into work before the clock runs out. The faster the hardware obsolesces, the more expensive idle time becomes, and the cadence is accelerating, not slowing.
The two ledgers are the same waste wearing different clothes. Rented, idle bills by the hour and shows up as cloud spend on the operating statement. Owned, idle bleeds value and shows up as depreciation against an asset earning nothing. Recovered utilization is the single lever that improves both: it turns metered hours into output for the renter and converts depreciating idle time into work for the owner. A customer does not have to decide which ledger to argue from. The same recovered point of utilization reads as recaptured cloud spend or recovered asset value depending only on how they hold the hardware.
The capital you do not spend
The third number is the one that reaches the board, and it lands the same whether the company rents or owns. In a normal market, low utilization is a reason to acquire more capacity, a bigger reserved-capacity commit for the renter, a new cluster for the owner. In this market, the next increment of net-new capacity is two to seven years away, gated by grid interconnect queues, transformer lead times, and turbine backlogs that money cannot compress. The next megawatt is not for sale on the timeline the business needs it, and that scarcity is feeding straight into cloud pricing and contract terms.
That changes what recovered utilization is worth. It is not a cost optimization sitting alongside the option to expand. It is the only capacity available on the business's timeline, because it comes from hardware already powered and racked. For the renter, that means the next reserved-instance renewal or net-new GPU commitment shrinks or slips, because existing contracted capacity is absorbing the work. For the owner, it is the difference between needing a second data center and not needing it, between a capital-expansion cycle and a deferral of one. Avoided spend of that size, whether a deferred cloud commit or deferred capex, does not show up as a small favorable variance. It shows up as a strategic decision the board makes or skips. Recovered utilization moves that decision from "when do we commit" to "do we need to yet."
Why the entry is cheap to approve
The last barrier to a finance yes is usually risk, the migration, the platform fight, the displacement of a working team. The utilization case is unusually clean on this axis, which is what makes it approvable.
The recovery happens at the execution fabric, below the orchestrator and above the silicon, so the scheduler, the clusters, and the pods stay exactly as they are. There is no rip-and-replace, no new platform to standardize on, no team to displace. The entry case is utilization recovered on hardware the customer already paid for, measured in their own numbers, on their own dashboards, before and after. That is the cheapest possible first yes for a finance owner: a benefit denominated in recovered capital, proved in the company's existing metrics, with the downside bounded because nothing above the execution fabric has to change to find out.
The point
Infrastructure spend usually asks a CFO to take an engineering benefit on faith. This one does not. It lands on three numbers finance already owns: the cost of the most expensive line in the compute budget, whether that line is a monthly cloud bill or a depreciating asset; the return earned on it today, about a twentieth of what is being paid; and the future commitment avoided by recovering the rest. The same recovered utilization reads as recaptured cloud spend for the renter and recovered asset value for the owner, and as a smaller next commit for both. Idle is not waste a company has to live with as the cost of doing AI. It is waste billing twice over, by the hour and off the balance sheet, and it is recoverable.
Capacity was the last decade's constraint, and you could buy your way past it. Fit is this decade's constraint, and the way past it is recovered utilization, the one infrastructure gain that reads the same in engineering and in finance, on the cloud bill and on the balance sheet alike.
A note on the numbers
The per-GPU-hour figures here are reported as ranges current to Q1 to Q2 2026, because GPU pricing has no single clearing price: the spread between specialized providers and hyperscalers runs 2x to 5x on the same hardware, and that spread is reported intact rather than collapsed to a flattering point. The worked unit economics ($3 per GPU-hour, a thousand-accelerator fleet) are framed as cloud spend because that is where most production capacity is billed, and they are explicitly illustrative and labeled as such; they exist to show the order of magnitude, not to quote a specific customer. The utilization figure is the same direct-measurement number used across this series, reported with what it measures. The depreciation debate is presented as an active, disputed accounting question, with the bull and bear positions both named, because it is not settled and should not be presented as if it were. As with the execution gap, the argument does not rest on any single number. It rests on the shape of all of them together.
References
- NVIDIA H100 at roughly $25,000–$30,000 (PCIe 80GB) and $35,000–$40,000 (SXM5 80GB); full 8-GPU DGX system exceeding $350,000; a complete production setup running 30–50% above the card price. CloudZero (May 2026). CloudZero
- H100 rental at roughly $2.00–$3.50 per GPU-hour on specialized providers, with a market median near $2.29–$3.12 across 40+ providers; specialized clouds 35–75% cheaper than hyperscalers. CloudZero; getdeploying (45+ providers). CloudZero getdeploying
- H100 on-demand at ~$6.88/hr on AWS and ~$12.29/hr on Azure (per GPU); 2–5x more expensive than specialized providers. Spheron (May 2026). Spheron
- Cast AI, 2026 State of Kubernetes Optimization Report. Average GPU utilization of 5% measured across tens of thousands of production clusters (AWS, Azure, GCP). Cast AI
- Independent reporting on the Cast AI findings: ~5% average across ~23,000 clusters, roughly 20x over-allocation. ITBrief. ITBrief
- Hyperscalers depreciating AI hardware over 4-to-6-year useful lives; Amazon shortening a subset of server lives while Meta extended in early 2025 under identical technology; analysts expecting convergence toward a 5-year cycle. National Law Review (Dec 2025); SiliconANGLE (Nov 2025). National Law Review SiliconANGLE
- NVIDIA shipping new architectures every 12–24 months against 6-year depreciation schedules; hyperscalers extending server life from 3-4 to 6 years, collectively saving ~$18B in annual depreciation. Introl (Jan 2026). Introl
- Michael Burry's estimate that stretched useful lives could understate industry depreciation by ~$176B across 2026–2028, overstating operating income at some operators by more than 20%, with a write-down or sustaining-capex "cliff" if real economic life is closer to 3 years. Levelheaded Investing (Dec 2025); Deep Quarry (Dec 2025). Levelheaded Investing Deep Quarry
- Microsoft CEO Satya Nadella: not wanting to be stuck with four or five years of depreciation on one generation, acknowledging obsolescence faster than accounting schedules. Stanley Laman Group (Nov 2025); Introl. Stanley Laman Group Introl
- Useful life used as an active earnings lever (the "Great Hyperscaler Divergence"); high utilization itself accelerating hardware degradation and shortening real useful life; A100 value erosion once H100 supply normalized. Stanley Laman Group; Cerno Capital. Stanley Laman Group Cerno Capital