Image generated with AI
Over the past year, a pattern has become hard to ignore in our work with enterprise customers. AI budgets that were approved with comfortable headroom are being used up faster than planned. At the same time, every industry update tells the same organisations that AI is getting cheaper. Both observations are correct, and the tension between them is worth understanding properly, because it changes how companies should buy and run AI.
Figure 1: Price per million tokens for a GPT-3-class model, 2021 to 2026, log scale. From $60 to $0.06.2
The numbers behind the "AI is getting cheaper" story are well documented. Stanford University's AI Index Report, one of the most widely cited academic references on the state of AI, found that the cost of querying a model at GPT-3.5 level fell more than 280-fold in under two years, from 20 dollars per million tokens in November 2022 to 7 cents by October 2024.1
Over a longer horizon the decline is even steeper. Industry pricing analyses that track models at a constant capability level find that a GPT-3-class model cost about 60 dollars per million tokens in late 2021, while the same capability was available for about 6 cents three years later, a factor of one thousand.2
Figure 2: Enterprise generative AI spend, $11.5 billion in 2024 to $37 billion in 2025.3
Menlo Ventures, which publishes one of the most cited annual surveys of enterprise AI adoption, puts enterprise spending on generative AI at 11.5 billion dollars in 2024 and 37 billion dollars in 2025, a 3.2-fold increase in a single year.3
This is not a contradiction, and it is not a failure of discipline. It is a textbook case of what economists call the Jevons effect: when a useful resource becomes cheaper, total consumption of it tends to rise, not fall, because far more applications become economically viable. Cheaper tokens did not shrink AI budgets. They made AI worth deploying everywhere at once.
Figure 3: The agent loop. A single request re-sends its accumulated context at every reasoning step and retries on error. The marked points are where a governance layer keeps consumption in check: caps and circuit breakers against runaway loops, routing that sends each task to the most economical capable model, and an evaluation gate that verifies quality.
A second effect matters more for the years ahead. The first wave of enterprise AI consisted mainly of chatbots: one question, one answer, one manageable amount of tokens. The current wave consists of agents, systems that plan, call tools, check their own results and retry when something fails.
This architecture has a direct consequence for cost. An agent re-sends its accumulated context with every step, so a single task can trigger ten or twenty model calls. Gartner estimates that agentic workloads consume 5 to 30 times more tokens per task than a standard chatbot query.4 For coding agents, researchers working with the SWE-bench benchmark have measured gaps of up to a thousandfold compared to simple code chat, driven almost entirely by input tokens the model re-reads at every step.5
The countermeasures are as structural as the problem. They sit around the loop, not inside the model, and in practice there are three of them: hard caps and circuit breakers that stop a runaway loop before it becomes an invoice, routing that sends each task to the most economical model that can handle it, and an evaluation gate that verifies the results still meet the quality bar. This set of controls is what we will call a governance layer in the rest of this article; Figure 3 shows where it attaches.
Our customers are already paying for the tokens. We propose and bring the solutions that turn more of them into results.
Figure 4: Token consumption relative to a chatbot baseline, log scale.4,5
In practice this means that an unsupervised agent can spend a very noticeable amount of money in a short time, and the invoice arrives later. Whoever operates agents at scale needs limits, monitoring and clear ownership of consumption. Very few organisations have assigned that ownership today.
Figure 5: Cost versus quality on a public coding leaderboard. Two models reach the same 44 percent score at $3.92 and $0.16 per task. Illustrative of a pattern visible across public cost and quality data.6
The third finding is the least known one. Model price does not reliably track model quality. On a public coding leaderboard we analysed this summer, two models reached the same 44 percent score, one at 3.92 dollars per completed task and one at 16 cents. The same pattern, price differences of an order of magnitude at comparable quality, is visible across public cost and quality data.6
Figure 6: Cost-reduction ranges reported across FinOps analyses and observed in our own projects.7
The consequence sounds simple: use the cheapest model that is good enough for the task at hand. The difficulty is that this decision has to be made thousands of times a day, task by task, and it has to be verified, because a cheaper model that quietly degrades quality is not a saving. This is why the practical answer is tooling: automated routing behind one gateway, hard budgets per workload, and evaluation gates that prove quality before and after a model change. Industry FinOps analyses and our own project experience put the achievable effect of these measures between 30 and 96 percent of addressed spend, depending on the lever and the starting point.7
Token prices will most likely continue to fall, and consumption will most likely continue to rise, with agents accelerating both trends. For enterprises, the conclusion is not to negotiate harder on unit prices. It is to give AI consumption an owner, the way cloud spend received an owner a decade ago: one gateway in front of all model usage, routing to the most economical suitable model, budgets and caps per workload, and continuous evaluation so that cost decisions never trade away quality unnoticed. For regulated workloads in Europe, jurisdiction belongs on that list as well, because for patient records or citizen data the binding constraint is not price but where and under whose law the processing runs.
Why GPUs are the engine of low cost per token: thousands of parallel cores turn the same workload into more tokens per second at a lower cost per token. The Industrial AI Cloud runs 1,000+ systems with ~10,000 NVIDIA Blackwell GPUs at 0.5 EFLOPS.8
At T-Systems we have built our AI platform around exactly this understanding, starting at the physical layer. Cheap tokens at agent scale are, in the end, a hardware question: a CPU works through a task with a few powerful cores in sequence, while a GPU spreads it across thousands of parallel cores. More tokens per second means a lower cost per token. GPU throughput is what makes agentic volumes and open-weight economics viable at scale.
Figure 8: Dedicated AI GPU capacity in German sovereign clouds. The Industrial AI Cloud in Munich adds roughly 50 percent to Germany's AI compute capacity.8
This is why we built the Industrial AI Cloud in Munich: roughly 10,000 NVIDIA Blackwell GPUs across more than 1,000 systems, which added about 50 percent to Germany's dedicated AI compute capacity when the site went live in early 2026.8
One site, Munich. Data stays in Germany, sovereign by design.8
And the location is not a detail. The site is German-operated, and the data it processes stays in Germany, which for regulated workloads is the deciding property, not the price per token.
On top of that capacity run the AI Foundation Services, our managed platform for language models, and the Agentic Hub, our implementation of exactly the governance layer described in Figure 3: it routes every task to the most economical suitable model, enforces budgets and caps for agents, runs the evaluations, and produces one auditable view of consumption per workload, under European jurisdiction. Our role in the token economy is not to sell cheaper tokens. It is to make sure our customers stop wasting them.
Model prices and total spending move independently. As the cost per token falls, far more use cases become economically viable, so organisations deploy AI more widely and consume far more tokens overall. Economists call this the Jevons effect. Enterprise generative AI spending rose from 11.5 billion dollars in 2024 to 37 billion dollars in 2025, even as inference prices collapsed.
Agentic systems consume between 5 and 30 times more tokens per task than a standard chatbot query. The reason is architectural: an agent re-sends its accumulated context with every reasoning step, and a single task can trigger ten or twenty model calls. For coding agents, measurements have shown gaps of up to a thousandfold, driven almost entirely by input tokens re-read at each step.
Not reliably. Price and quality do not track each other closely. On a public coding leaderboard, two models reached an identical 44 percent score at 3.92 dollars and 16 cents per completed task. Choosing the cheapest model that meets the quality bar for each task is therefore one of the largest cost levers available, provided quality is independently verified.
A governance layer is the set of controls placed around model usage rather than inside the model. In practice it has three parts: hard caps and circuit breakers that stop runaway agent loops before they become an invoice, automated routing that sends each task to the most economical capable model, and evaluation gates that verify quality before and after any model change.
In a sovereign setup, processing location and legal jurisdiction are defined properties rather than side effects. The Industrial AI Cloud in Munich is German-operated, and the data it processes stays in Germany. For regulated workloads such as patient records or citizen data, this is often the binding constraint on model choice rather than the price per token.
1 Stanford HAI, AI Index Report 2025. Inference cost at GPT-3.5 level down more than 280-fold, November 2022 to October 2024.
2 Menlo Ventures, "2025: The State of Generative AI in the Enterprise". Enterprise GenAI spend $11.5B (2024) to $37B (2025).
3 Gartner estimate, as reported in Computerworld, "AI inference is getting cheaper, but your agents are getting more expensive". Agentic workloads at 5–30× chatbot token consumption.
4 Deutsche Telekom / T-Systems and NVIDIA announcements on the Industrial AI Cloud: telekom.com, t-systems.com, blogs.nvidia.com. ~10,000 Blackwell GPUs; adds ~50% to Germany's AI compute capacity.
5 SWE-bench coding-agent measurements, reported in the same Computerworld analysis: up to ~1000× versus simple code chat, driven by re-read input tokens.
6 Public coding-leaderboard data, August 2026; the specific 24.5× pair is illustrative. The broader pattern is visible in public cost-versus-quality datasets such as Artificial Analysis.
7 FinOps Foundation, finops.org, working guidance on optimising GenAI usage, combined with T-Systems project experience. Ranges depend on lever and baseline.
8 Deutsche Telekom / T-Systems and NVIDIA announcements on the Industrial AI Cloud: telekom.com, t-systems.com, blogs.nvidia.com. ~10,000 Blackwell GPUs; adds ~50% to Germany's AI compute capacity.
Figures are redrawn from the cited values; Figure 5 is illustrative. Capacity claim is "adds roughly 50 percent to Germany's dedicated AI compute", not half of Germany's GPUs.