The pilot invoice is a sample size of one
Compute got 280 times cheaper. Compute bills went up. Both are true, and the gap between them is where AI programs go to die.
Stanford HAI measured the cost of a standard AI query falling roughly 280 times in 18 months, from $20 to $0.07 per million units of text (AI Index Report 2025). IBM's Institute for Business Value projected that enterprise compute costs would rise 89% between 2023 and 2025, found that cloud infrastructure already costs about twice as much as the model licenses, and reported that 21% of generative AI initiatives fail because of compute cost (The CEO's Guide to Generative AI: The Cost of Compute, 2024). Every AI leader IBM surveyed had canceled or postponed at least one initiative over compute costs. Yes, every one. A defined sample, not the whole market, but nobody in it was spared.
What this means for an owner: these numbers only look contradictory if cost per unit is what governs your bill. It is not. What governs the bill is a number almost no pilot produces: what a production month costs.
What a pilot invoice actually measures. A pilot is a few dozen users, curated data, forgiving response times, no peak load, no redundancy, and none of the integration or security overhead production brings. Its invoice is a sample size of one, a single lit lamp in a large house: accurate about that lamp, silent about the meter when every room is on. The pilot was not wrong. It was never asked the question.
Position A: prove the return first. Commit later.
This view deserves its full weight. Unit prices are collapsing; locking in infrastructure at today's prices is how you overpay for tomorrow's capacity, and buying ahead of evidence creates stranded assets. The commercial pressure is real too: a pilot that does not show value in a quarter does not get a second quarter. Rent flexible capacity, move fast, build the production architecture once the use case has earned it. Speed to demo is not a character flaw. It is how budgets get unlocked.
Position B: price production before you pilot.
The opposing view starts from where the money actually dies. Cancellations do not happen in the pilot. They happen at scale-up, when volume, concurrency, and retries meet a cost model nobody wrote down, after the executive promise and the hiring. By then a CFO is deciding with the least information at the worst moment. The 21% and the 100% (of surveyed leaders) are not technology failures. They are forecasting failures. And the longer the forecast waits, the more the fix costs: governance, identity, security, and capacity bolted onto a finished pilot cost materially more than building them in. A multiple, not a margin. That is the retrofit tax, and it is charged by sequence, not by any vendor.
The synthesis: defer the commitment, not the forecast
Both camps are right about different things. A is right about spend: do not buy capacity ahead of evidence. B is right about information: do not scale ahead of a cost model. The error is treating them as one decision. Pilot on rented, flexible capacity, and still write down, before the pilot starts, what its production month would cost. Fast pilots, honest forecasts. The trap is speed to demo versus rigor to production; the way out is letting the demo be fast while refusing to let its invoice stand in for the cost model.
Cheaper units do not shrink the bill. They widen what is affordable to attempt (bigger requests, more automated agents, more retries), so volume rises to meet the price. That is how a 280 times price drop and a projected 89% cost rise coexist. Unit price is the vendor's metric. Total cost is yours.
Where this discipline already exists
I know what you are thinking: my business does not run that way, and I do not have a platform team. Fair. Here is why I raise it anyway. In my operating career I have led production platforms where capacity was provisioned, secured, and approved before a workload was admitted, so "what does this cost at full load, and where does it run?" was answered before the first user logged in, not after the first invoice. Production was not phase two. The workload was born in production.
The version that fits an established business is not the same rigidity. It is the same honesty about what production requires. You keep the flexibility. You just stop pretending the pilot priced the program.
What this means for leaders
1. The Day-Two Bill. Before a pilot is approved, write the production month on one page: users, usage per user, size of each request, peak concurrency, redundancy, and integration and security overhead, priced on the infrastructure you would actually run it on. Put it next to the pilot invoice. The ratio between them is the number the board never sees. If you cannot write the page, you have learned something cheaply: the pilot is not ready.
2. Cost per accepted outcome. Retire cost per unit as a headline metric; it falls whether or not your economics improve. Track the cost of each task a human accepted. Retries, rework, and review live in that denominator. If unit price keeps falling and your cost per accepted outcome does not, the savings are being consumed upstream. This is the board-level number.
3. The pre-committed kill line. At kickoff, write down the production run-rate and cost per outcome at which the program is paused or re-scoped, and name who decides. If every surveyed leader eventually cancels or postpones something, the only open question is whether cancellation arrives as a decision or as an ambush.
The CFO perspective
Pilot spend is operating expense that buys no asset. Production run-rate is a standing commitment against margin. The CFO will pull the plug regardless of which vendor you chose, in the quarter the run-rate stops being explainable. The Day-Two Bill turns that conversation into a forecast you made rather than a surprise you explain.
The window
It closes quietly: after the pilot proves out, before the first production invoice. Falling unit prices tempt owners to treat cost as a problem the market will solve. It will not solve this one. Write the bill down while it is still a document and not an invoice. If you want to pressure-test yours, comment DAYTWO on Thursday's thread and I will send the one-page Day-Two Bill worksheet, yours whether or not we ever work together.
