Why Buying More Capacity Is Wrong

The argument for buying more, stated fairly

The case for more capacity is not weak, and it should be put plainly before it is answered.

Demand for compute is compounding, and the mechanism changed in a way that favors scale. For the first years of the build-out the heavy spend went to training, a finite event per model. Reasoning models moved the load to inference and made it continuous. A reasoning completion generates thousands to tens of thousands of internal tokens before it answers, and a single hard query can require on the order of 100x the compute of one pass through a conventional model.¹² NVIDIA's position is that test-time scaling drives further demand for accelerated computing, not less.² The load is no longer a capex cycle that crests and falls. It behaves like an industrial base that has to stay on.³

The numbers are large and real. NVIDIA now sees more than a trillion dollars in Blackwell and Rubin chip demand through 2027, a figure it doubled inside a single year, and its CEO has put the eventual capex envelope at three to four trillion dollars.³⁴ The fourteen largest data center operators are heading toward roughly 750 billion dollars of capex in 2026, up from under 450 billion the year before.⁵ Forecasts for the build-out have been revised up quarter after quarter and have run behind the actual spend, not ahead of it.⁶ When the people closest to demand keep underestimating it, more looks like the safe bet.

There is also a direct answer to the efficiency argument. The frontier does not depend on any one operator's utilization rate. Capability comes from scale, and the lab that controls the most capacity sets the pace regardless of how tightly it is packed. On this view, owning the capacity is the advantage and optimizing it is housekeeping. The fiber build-out of the late 1990s is the cited precedent: capacity that looked excessive at the time created enormous value later.⁷

Most of this is correct. Demand is compounding. The frontier runs on scale. The forecasts have been too low. The conclusion still does not follow, for three reasons.

You use about five percent of what you buy

The argument for buying more assumes that the work you get out of a fleet tracks the capacity you put in, that 2x the peak gives you something close to 2x the work. That assumption is false by a wide and growing margin.

Peak compute per chip rose roughly twentyfold across a single hardware generation, from 125 dense FP16 TFLOPS on a V100 to 2,500 on a GB300.⁸ The share of that peak turned into useful work did not move with it. Even a well-tuned training run reaches only 35 to 45 percent of theoretical peak.⁹ Mixed production fleets do far worse. Direct measurement across tens of thousands of clusters puts average GPU utilization near 5 percent, which means the buyer is paying for roughly twenty times the capacity it puts to work.¹⁰¹¹

The gap is not a rounding error. It is the majority of the asset, and it widens as peak grows, because a flat utilization rate multiplied against a peak that grew twentyfold is twentyfold more wasted compute per chip. Buying more peak when you use 5 percent of the peak you have does not buy more work. It buys more idle capacity at full price.

The reasoning shift is a fit problem, not a capacity problem

The strongest point in the case for buying more, the move to reasoning and inference, points the other way once you look at what it does to the hardware.

A reasoning query is decode-heavy and autoregressive. Every internal token requires reading the entire growing key-value cache from memory, so the bottleneck inside the GPU shifts from raw compute to memory bandwidth and capacity.¹ The prefill phase of inference is compute-bound and the decode phase is memory-bandwidth-bound, and running them on the same accelerator means one phase is starved while the other saturates. This is structural, not a tuning error.¹² The field has converged on the same fix: separate the phases and run each on the resource it needs. NVIDIA shipped Dynamo to general availability in March 2026 to do exactly this, reporting up to roughly 7x throughput on Blackwell from the separation, and Google Cloud publishes a recipe on the same principle.¹²¹³

So the reasoning era does increase demand for compute. It also makes each unit of compute more sensitive to how well the work is matched to the resource, because the work now has sharply different resource profiles inside a single query. More demand for compute and more demand for fit are the same trend, not opposing ones. The inflection everyone points to runs toward the execution fabric.

You cannot buy your way past the power wall

The case for buying more carries a last hidden assumption inside the word buy: that capacity bought is capacity online. For most of the cloud era that held. It no longer does.

New capacity is gated by power, not by willingness to spend. The US interconnection queue holds roughly 2,600 GW of proposed generation waiting for grid access, more than double the country's entire existing operational capacity, with waits of four to ten years for projects that need real transmission work.¹⁴ Large power transformers run well beyond 24 months of lead time, with specialized units stretching to 36 to 48 months.¹⁵ Large-frame gas turbines from the three makers that supply them are effectively booked through 2028.¹⁶ Microsoft has reported restricting new Azure capacity in hubs including Northern Virginia and Texas because it lacks the power to bring capacity online, not the demand to fill it.¹⁷

A company can sign a purchase order for GPUs in an afternoon. It cannot buy a substation, an energization date, or a jurisdiction willing to absorb concentrated load on the same clock. When the constraint is physical, the only capacity that arrives on demand is the capacity already powered and running at 5 percent. Recovering it is not the cautious alternative to buying more. In a market where the next megawatt is years out, it is the faster path to more usable compute.

The economics already say wait

The buying-more case does not even require the spending to be irrational to be premature. The returns are not in yet, and the people defending the build-out say so.

By Q1 2026 the run-rate AI revenue across the hyperscalers sat well below the spend, with the ratio of AI capex to AI revenue projected to hold in the 45 to 57 percent range before improving only as revenue scales.¹⁸ Sequoia's widely cited framing put the gap between required and actual AI revenue in the hundreds of billions.¹⁹ Defenders of the build-out concede it is a time-lag bet, that infrastructure laid today may take 18 to 36 months to earn its return.²⁰ During that wait, the lever that pays now is the one that turns capacity already bought into work already billable. That is fit, on hardware the customer already owns.

The point about reinvestment holds, but it favors fit. As fit makes delivered compute cheaper to run, the work that was waiting on capacity gets run, and the layer that converts bought capacity into delivered work captures that demand. The reinvestment runs through the execution fabric, not around it.

The point

The case for buying more is right about demand and wrong about the lever. Demand is compounding, the frontier runs on scale, and the forecasts have been too low. But delivered work does not track installed capacity. It trails it by a factor of roughly twenty, the factor grows as the hardware grows, the reasoning shift makes fit matter more rather than less, the power wall closes the buy-more exit, and the returns are years out. Capacity will keep being bought. The returns will go to whoever turns the most of it into work.

A note on the numbers

The figures here are reported with their source and what they measure, and the projections are labeled as projections. The demand and capex figures, the trillion-dollar chip forecast, the 750 billion dollar operator capex, the three-to-four-trillion envelope, are vendor and analyst forward estimates. They are presented as the strongest evidence for buying more, not disputed, because the argument does not depend on them being wrong. Utilization figures distinguish fleet utilization, model-FLOPs utilization, and server-capacity utilization, which are three different things. The reasoning-compute multiplier of roughly 100x for hard queries and the Dynamo throughput figure are vendor-reported on specific workloads and labeled as such. The capex-to-revenue ratios are Q1 2026 estimates current to that quarter. As across the series, the argument does not rest on any single number. It rests on the shape of all of them together: a peak that grew twentyfold, a delivered share that did not, and a physical wall that closes the buy-more exit.

References

1. Reasoning completions generate thousands to tens of thousands of internal tokens; decode-heavy, autoregressive work where each token reads the growing KV cache, shifting the bottleneck from FLOPs to memory bandwidth and capacity; 30 to 100x the inference compute of a comparable chat query. Towards Data Science (May 2026); Medium analysis of test-time compute economics (May 2026). Towards Data Science Medium
2. NVIDIA's position that test-time scaling requires intensive compute and drives further demand for accelerated computing; reasoning can require over 100x compute for hard queries versus a single pass. NVIDIA Blog on scaling laws. NVIDIA
3. Huang's GTC 2026 framing that token-producing compute is an always-on industrial base rather than a temporary capex cycle, anchoring the trillion-dollar forecast. eWeek coverage of GTC 2026. eWeek
4. NVIDIA seeing more than a trillion dollars in Blackwell and Rubin demand through 2027, doubled within a year; Huang putting the eventual capex envelope at three to four trillion dollars. Fortune (Mar 2026); CNBC (May 2026). Fortune CNBC
5. The fourteen largest data center operators heading toward roughly 750 billion dollars of capex in 2026, up from under 450 billion in 2025; over 23 GW under construction. BloombergNEF (Mar 2026). BloombergNEF
6. Analyst capex forecasts revised upward quarter after quarter and running behind actual spend; FY2027 expectations for the largest developers climbed 56 percent between August 2025 and February 2026. BloombergNEF; io-fund on the pattern of under-forecasting. BloombergNEF io-fund
7. The fiber-optic build-out of the late 1990s as the historical precedent: capacity that looked excessive at the time ultimately created enormous value, though it punished early investors with overcapacity. Tech Insider analysis of the 2026 capex race. Tech Insider
8. Dense FP16 peak per chip rising from 125 TFLOPS (V100) to 2,500 (GB300), roughly twentyfold in a single generation. NVIDIA H100 datasheet; GB300 NVL72 per-precision figures via Verda; V100 figure via Spheron. NVIDIA Verda Spheron
9. 35 to 45 percent MFU on well-tuned training runs. Meta Llama 3 (arXiv 2407.21783); CoreWeave H100 benchmarks. arXiv CoreWeave
10. Cast AI, 2026 State of Kubernetes Optimization Report. Average GPU utilization of 5 percent across tens of thousands of production clusters (AWS, Azure, GCP), with both CPU and GPU down year over year. Cast AI
11. Independent reporting on the Cast AI findings: roughly 5 percent average across about 23,000 clusters, roughly 20x over-allocation. ITBrief. ITBrief
12. Prefill compute-bound, decode memory-bandwidth-bound; co-location causes interference. NVIDIA Dynamo reached general availability at GTC on March 16, 2026, shipping prefill/decode separation with up to roughly 7x throughput on Blackwell. NVIDIA Dynamo design docs and product page. NVIDIA docs NVIDIA
13. Google Cloud disaggregated-inference recipe using NVIDIA Dynamo on AI Hypercomputer. Google Cloud
14. US interconnection queue near 2,600 GW of proposed generation and storage, more than double existing operational capacity; waits of four to ten years for projects needing transmission work. Futurum; AL Capital Advisory (CFA analysis, May 2026). Futurum AL Capital Advisory
15. Large power transformer lead times beyond 24 months, with specialized units at 36 to 48 months. AL Capital Advisory. AL Capital Advisory
16. Large-frame gas turbines from the three primary makers effectively booked through 2028. AL Capital Advisory; corroborated in the energy-constraint reporting underlying the series. AL Capital Advisory
17. Microsoft restricting new Azure capacity in hubs including Northern Virginia and Texas for lack of power, not lack of demand. Futurum. Futurum
18. Q1 2026 AI capex-to-revenue ratios projected to hold in the 45 to 57 percent range before improving as AI cloud revenue scales; required run-rate AI revenue well above current run-rate. AL Capital Advisory (CFA analysis). AL Capital Advisory
19. Sequoia's framing of the gap between required and actual AI revenue, measured in the hundreds of billions. ToolDirectory summary of the Cahn/Sequoia analysis. ToolDirectory
20. Buildout defenders conceding the time-lag bet: infrastructure built today may take 18 to 36 months to generate proportional returns. Futurum. Futurum