AI Is Getting Cheaper to Build — So Why Is Enterprise AI Still So Expensive?

The price of using AI keeps falling.

Models are getting faster, smaller models are becoming surprisingly capable, and cloud providers keep finding ways to lower the cost of inference. Google is currently offering Gemini 3.7 Flash at introductory rates of $0.75 per million input tokens and $3.75 per million output tokens, while companies can increasingly choose between premium reasoning models and cheaper models for routine work.

From the outside, the economics seem obvious: if models become cheaper, enterprise AI should become cheaper too.

Yet many companies are experiencing the opposite.

Google recently introduced new billing and cost-control tools specifically because businesses are struggling to understand and manage the cost of agentic AI. In one case described by Google Cloud, a customer’s AI spending increased by more than 50% in a single month, and the company could not clearly identify which team, project or agent was responsible.

This is not really a contradiction.

The problem is that the price of a model call and the cost of running AI inside a company are becoming two very different things.

The Token Price Is Only the Beginning

When generative AI first entered the enterprise, calculating costs was relatively straightforward.

A user asked a question. The model processed some input tokens, generated an answer, and the company paid for that inference.

Agents change that equation.

Ask a traditional chatbot to analyze a contract and you may have one model interaction. Give an agent responsibility for reviewing contracts across a workflow and it may need to retrieve documents, search internal systems, call several tools, reason over the results, correct failed attempts, update a database and ask another model to verify its work.

One business task can suddenly involve dozens or even hundreds of model calls.

Microsoft recently described exactly this problem while introducing TokenOps, an experimental system for controlling agent spending. A single agent run may make hundreds of calls, and retries can quietly accumulate when a tool returns a weak result or the agent fails to make progress. Each individual call looks inexpensive. The total run may not be.

This is why looking only at cost per million tokens can be misleading.

A cheaper model helps, but the bill ultimately depends on questions such as:

How often does the agent call the model?

How much context does it send every time?

How many tools does it invoke?

How often does it retry?

Does another agent review the result?

Does it use a premium model for every step or only the difficult ones?

How long does it keep working before deciding that it is stuck?

Once AI starts completing workflows instead of answering prompts, the unit of cost shifts from the token to the task.

That is a much more useful way for companies to think about AI economics.

The Model Is Getting Cheaper. The System Around It Is Getting Bigger.

There is another reason enterprise AI bills do not fall as quickly as model prices.

Production AI needs much more than a model.

Google’s own pricing for its enterprise agent platform makes this unusually visible. Companies may pay separately for agent compute, memory storage and operations, session state, model tokens and governance functions. Memory generation and embeddings create their own token usage. Policies designed to inspect an agent’s actions may involve additional model evaluations.

None of these things are unnecessary extras.

They exist because an AI that does real work needs to remember what happened, connect securely to business systems, maintain state across tasks and operate within company rules.

Then there are the costs that do not appear neatly on an AI API invoice.

Enterprise data needs to be cleaned and connected. Permissions have to be designed. Security teams need to assess new risks. Outputs may require human review. Agents need monitoring. Failed workflows need debugging. New evaluations have to be written whenever the system changes.

And the more important the use case becomes, the more expensive these surrounding controls tend to be.

A marketing assistant that drafts social posts can tolerate occasional mistakes.

An agent that changes prices, processes a payment or reviews a legal agreement cannot.

That is why Google is emphasizing governance beneath its new industry-specific AI products for financial services and legal work. Model intelligence alone is not enough in environments where data lineage, permissions and confidentiality matter.

Enterprise AI is therefore following a familiar technology pattern.

The core component becomes cheaper.

So companies use much more of it.

And as usage expands into more valuable work, the infrastructure around that component becomes more sophisticated.

Cheap compute does not automatically create a cheap system.

Agents Make AI Spending Harder to Predict

Traditional enterprise software is relatively easy to budget.

A company buys 1,000 licenses at a known price per employee. Usage may vary, but the finance team has a reasonable idea of what the software will cost next quarter.

Agentic AI behaves differently.

The cost of completing the same type of task can change depending on complexity. One customer request might require three reasoning steps; another may require thirty. One coding task may finish immediately; another may trigger several retries, searches and debugging loops.

Microsoft’s Azure Observability Agent already prices deeper investigations differently because they require more agent and tool work than simple questions.

This variable nature of AI is pushing cloud economics toward something closer to FinOps than traditional SaaS procurement.

Google’s latest response is telling. On August 26, it expanded billing options for Gemini Enterprise and introduced additional controls designed to give companies visibility into agent spending and protect budgets. The company is explicitly framing this as FinOps for the AI era.

That may turn out to be a larger shift than it sounds.

Companies spent years learning how to optimize cloud infrastructure: tagging resources, allocating costs to teams, setting budgets and finding unused capacity.

Now they may need to develop a similar discipline for AI.

Not simply:

How many tokens did we use?

But:

Which workflow used them, what business result did it produce, and was that result worth the cost?

That last question matters most.

An agent that costs $20 to perform a task that saves an employee four hours may be extremely cheap.

An agent that costs $2 but repeatedly performs work nobody needs is expensive.

The right metric is not cost alone.

It is cost per useful outcome.

The Cheapest Model May Not Produce the Cheapest Workflow

This also complicates one of the most common AI cost-cutting strategies: simply switching to a cheaper model.

Sometimes that works.

But imagine a low-cost model needs five attempts to complete a task that a stronger model finishes in one. Or it produces more errors, requiring another model or a human to review the result.

The cheaper model can easily create the more expensive workflow.

The opposite can also happen. Using an expensive frontier model for routine classification, extraction or summarization can waste money when a smaller model would perform the job just as well.

The efficient architecture is likely to use several levels of intelligence.

Cheap models handle predictable work.

More capable models are called when the task becomes difficult.

Humans step in when the cost or risk of a mistake becomes high.

This is one reason the AI infrastructure market is increasingly focused on routing, observability and cost control rather than simply selling more tokens. Google has even pushed back against what it calls “tokenmaxxing” — the idea that throwing more context, reasoning or compute at a problem automatically produces a better business result.

More intelligence has value. But only when the value exceeds the additional cost.

Enterprise AI May Need a Different Definition of “Cheap”

There is a temptation to compare AI with traditional software and ask when prices will fall enough for every company to use it everywhere.

That may be the wrong question.

OpenAI’s latest enterprise data suggests leading companies are already moving from using AI for assistance toward delegating entire pieces of work to agents. What separates those companies is not access to a unique model; many organizations can buy access to the same underlying technology. The difference is how deeply they connect AI to tools, context and actual workflows.

If AI stays as a chatbot, cheaper tokens probably translate fairly directly into lower costs.

If AI becomes labor — researching, coding, analyzing, operating systems and completing workflows — then its economics begin to look different.

The useful comparison may no longer be:

How much does this AI cost compared with another AI?

It becomes:

How much does this workflow cost with AI compared with doing it another way?

That is a much higher bar, but also a much more meaningful one.

A company could spend twice as much on AI next year and still become dramatically more efficient if those agents replace enough repetitive work, shorten enough processes or help employees produce more valuable output.

Conversely, falling token prices will not rescue an AI deployment that creates little measurable value.

That is why the next enterprise AI race may not be about who gets inference costs closest to zero. It may be about who becomes best at answering a more difficult question:

What did we actually get for every dollar of AI we spent?

As models continue getting cheaper, that question will become harder to ignore.

The technology is becoming less expensive. Using it well is not.

[Source]