If You Can't Calculate Your AI Cost Per Task, You Haven't Priced It
Enterprises are budgeting for AI as if it were per-seat software. It's a metered utility, and almost nobody on the buying side models the meter.
Most organisations buying AI at scale still cannot calculate their AI cost per task. They know the per-seat price of every other tool they own, and they know the headline rate on the vendor's pricing page. What sits between those two numbers, the cost of producing one finished piece of work, is usually a shrug and a monthly invoice that nobody can decompose.
That gap isn't a finance-team detail, it's a category mistake. Business software has been sold per seat for thirty years, so procurement instinctively models it per seat: one line on the forecast, scaled by headcount. What you've actually bought is a metered utility. The meter counts tokens, and the number of tokens a task consumes has almost nothing to do with how many people are logged in.
The meter measures something the seller defines
A token is not a word or a syllable or anything a human would recognise as a linguistic unit. It's a fragment produced by a compression algorithm that was repurposed as a vocabulary. The technique now underpinning the industry's billing was adapted from a byte-pair-encoding compression method by Rico Sennrich, Barry Haddow and Alexandra Birch at the University of Edinburgh, building on Philip Gage's 1994 compression work. It was chosen because it handled rare words gracefully, and nobody picked it to be a unit of account. It became one anyway.
The buyer signs a contract denominated in a quantity that is an artefact of a training run, while the seller publishes the price per million of them on a live public pricing page that can change. Rate cards are visible in a way volume never is. You control neither how many tokens your prompt becomes nor how many the system decides to emit in reply, and the invoice is the product of the two.
How do you calculate AI cost per task?
Start with the mechanism: the model has no memory between requests. Continuity is an illusion produced by resending the entire conversation, plus hidden system instructions and tool definitions, on every turn. Vendor documentation is explicit that previous-turn outputs become inputs on later turns.
The arithmetic follows without needing a benchmark. If each turn adds roughly k tokens and every accumulated turn is processed again, total processed context across n turns is k(1+2+…+n), which is kn(n+1)/2, quadratic in turn count. For example, if a session runs ten turns and each adds about a thousand tokens, you're billed for roughly 55,000 input tokens rather than the 10,000 a linear mental model predicts, because the tenth exchange pays for the first nine all over again. A chat assistant answers once and stops; an agent loops, and the loop is the cost curve.
Now add the fixed overhead. System instructions and tool schemas ride along on every call, so short tasks carry a terrible overhead ratio: you can spend most of a request re-describing the tools rather than doing the work. Then divide by the success rate. For example, if seven attempts in ten produce a usable outcome, your true cost per completed task is total spend divided by 0.7, some 43% above the naive figure.
The meter runs on attempts while value accrues only on successes, and any AI business case that quietly assumes those two are the same number is overstating its margin.
Why does the same task cost more in some languages?
Because the vocabulary was learned, mostly from English-heavy web text, and it compresses what it saw often far better than what it saw rarely. Independent work presented at NeurIPS by Petrov and colleagues, comparing parallel multilingual text, documents substantial language-dependent differences in tokenised length, with the magnitude depending on the tokeniser and the language pair.
What that produces is a margin structure nobody chose. The same customer service interaction, delivered to the same standard, costs the business systematically more in some markets than others, and the difference is invisible in any dashboard reporting cost per user. Multilingual products have a per-market unit economics problem that never made it into the pricing model, and code-heavy workloads inherit the same asymmetry from the other direction.
The ceiling that fails silently
The context window is a hard limit, and overrunning it means something gets dropped. The symptom, degraded answers deep into a long session, looks exactly like unreliability, so teams escalate to a bigger model or a different vendor when what they had was a capacity overrun they provisioned wrong. Diagnosing that requires knowing the published limits of what you bought, and published means vendor-stated and dated, in the manner of OpenAI's May 2020 paper identifying GPT-3 as a 175-billion-parameter autoregressive model. Anything sourced from a blog aggregator is folklore. This is the sort of thing worth settling before you commit to a build, not during the incident review.
What breaks when the discounts end
The obvious objection: token prices have fallen repeatedly, so why model any of this? Because unit price and units consumed are moving in opposite directions. Per-token rates drop while agentic patterns push consumption per task up by an order of magnitude, and the second effect is currently outrunning the first. Falling prices are an argument for knowing your curve, not against it. The curve tells you which workloads flip from unviable to viable at which price, and which ones were only ever solvent because someone else was absorbing the difference.
Whether today's rates sit above or below the cost of serving them is not something buyers can verify from outside. The structure is visible enough: the seller defines the unit, sets the rate, and revises both, while the buyer has no independent way to audit the quantity billed. That's an unusual amount of asymmetry to accept in an operating line item, and the sane response is engineering rather than complaint: measure tokens per completed task per workload, instrument the failure rate, cap conversation length deliberately, and design agent loops that don't drag their entire history behind them. Those are architecture decisions about how agents are built, and they belong in the technical strategy, not the finance review.
A firm that can quote its AI cost per seat, per call, or per month has measured something. It just hasn't measured the thing it's buying.
Questions people ask
What should I measure instead of tokens per API call?
Tokens per completed task, broken down by workload, with the failure rate attached. A per-call figure hides both the resubmitted context of multi-turn sessions and the cost of attempts that produced nothing usable, and those are the two places overspend actually lives.
Does caching or shortening prompts fix the super-linear cost of long agent sessions?
It reduces the constant, not the shape of the curve. Trimming system instructions or reusing cached context lowers the per-turn cost, but as long as accumulated history is resubmitted each turn, total processed context still grows with the square of the turn count. Bounding session length and summarising history changes the curve itself.
Why does our AI spend rise without more users being added?
Because spend tracks work done rather than seats occupied. A single user running longer, more agentic tasks with more tool calls will consume far more than several users asking one-shot questions, so headcount-based forecasts drift almost immediately once agents replace chat.
Related
- The Memory Oligopoly Behind AI's Cost Inflation
- Least privilege for AI agents is a 1975 idea your pilot is skipping
- The AI Subsidy Cliff: Your Vendor's Investors Have Been Paying Your Bill
- AI & Automation
Written by an AI editorial persona of Abyshire's proprietary editorial system and reviewed by our team.