Insights

Why Are Companies Blowing Through a Year of AI Budget in Months?

The stories have become familiar. A company budgets for AI annually and exhausts the allocation by the end of the first quarter. Boards that spent last year asking for AI are now asking what it returned.

The spending itself is not the surprise. What catches organizations off guard is where it comes from, because very little of it shows up as the line item they approved. That was the second half of a recent episode of the Amplitude of Tech podcast, in conversation with Bryan Smith, CEO of Expedient, an intelligent infrastructure provider offering private cloud, data center, and AI as a service. Smith broke down why consumption is outrunning forecasts, how shadow AI and unmanaged agentic workflows drive spend invisibly, and what observability and least cost model routing actually look like in practice.

Quick Answer AI budgets get exhausted early because organizations price AI as a software license when it behaves like consumption-based infrastructure. Most are in a build phase with high implementation usage while budgeting for a steady run phase. Volume growth, not per-token price increases, is the dominant driver, amplified by shadow AI, unscoped agentic workflows, and default routing to the most expensive available model. Controlling AI spend requires observability with enforceable controls, not just dashboards, plus deliberate model selection for each task.

You’re Budgeting for the Run Phase While You’re in the Build Phase

Smith’s core diagnosis is a phase mismatch. Organizations are in a build phase, with the material costs that implies, while budgeting as though they are in a steady run phase.

His analogy: building a house consumes vastly more material than owning one and making occasional updates. A large software deployment carries an implementation fee alongside the license. AI gets approached as if only the license exists.

User profiles compound this. By Smith’s estimate, someone using AI to write software consumes twenty to a hundred times more than an average employee using a chat interface. Segmenting users by workload before forecasting is the step that gets skipped.

Volume Is the Real Driver, Not Price

It is tempting to attribute overruns to vendor price increases. The picture is more complicated.

Smith pointed out that today’s frontier models cost roughly three times more per use than the best model available a year ago, while the same level of capability from eighteen months ago now costs over 200 times less. Capability per dollar has improved dramatically even as the top of the market got more expensive. What actually outpaced everything is the number of users and the volume they consume.

Two genuine cost pressures do exist:

  • Hardware. Smith cited roughly 400 percent price increases on servers compared with eighteen months prior, with similar movement in GPUs, flowing through to infrastructure costs.
  • The end of subsidized flat rates. Flat-rate plans are heavily subsidized against API pricing. Smith noted that a two hundred dollar monthly plan can represent token volumes that would cost thousands through the API directly. Microsoft Copilot moving its development tier from flat rate to consumption was an early signal, and he expects consumption pricing to become the norm.

That shift is the reason observability stops being optional. Flat rates hid consumption. Consumption pricing exposes it.

Where the Invisible Spend Actually Comes From

Four sources, in Smith’s account, account for most of the surprise.

Configuration habits

Proper prompt caching on input tokens can cut consumption by as much as ninety percent, though it does nothing on the output side. Batch processing, for work that does not need a real-time response, runs roughly fifty percent cheaper on public models. Bloated context windows and weak prompt design add to the bill.

Model defaulting

Smith’s comparison: you do not visit the area’s best surgeon to have your temperature taken for a cold. Yet users default to the most expensive model, and most consumer AI platforms default that way too, because it produces the best experience. He described agent runs where swapping a top-tier model for a smaller one on appropriate steps cut cost to a fraction of the original.

Unscoped agentic workflows

An automation configured without regard to API limits or thresholds can consume an annual allocation in minutes. Smith’s framing is to stop calling them agents and think of them as digital employees. Employees get management, oversight, direction, and coaching, and none of that is possible without the ability to inspect their work.

Shadow AI

Smith considers the majority of AI work happening today to be shadow AI. Most organizations have not given employees the breadth of tooling they need, so people buy their own subscriptions and build on platforms a CIO never approved. He was explicit that this is not malicious. The business has not kept pace with providing the platform, the guardrails, or the enablement, so people use whatever they find. His dog park analogy fits: a fenced park lets the dog run safely because the boundaries exist, while an open door into the woods offers the same freedom with none of the protection.

A Dashboard Is a Speedometer, Not a Brake

Smith’s sharpest point concerns what most organizations call observability. A speedometer tells you your speed without limiting it. Most AI dashboards report last month’s spend without being predictive or real time.

Useful AI cost management adds enforceable controls on top of visibility:

  • Limits scoped by individual, team, agent, or role-based access control
  • Notifications routed to Teams or Slack, or into a ticket, when a threshold is crossed
  • Throttling that degrades performance gracefully instead of causing an outage
  • Automatic fallback from an expensive model to a cheaper one
  • Step-level replay of an agent run, so you can see which models were called, how large the prompts were, and which steps were unnecessary

Smith described building an agent inside the governance layer itself: when a team hits fifty percent of its allocation twenty percent of the way through the month, the business owner gets a message with options to continue, adjust, or switch models. That same replay data can be handed back to an agent for optimization recommendations as new models ship.

He estimated only twenty to twenty-five percent of companies have reached a mature approach, with sixty to sixty-five percent still working out what AI means for the business. His view on why: most tried to build the observability layer themselves. His comparison is buying a car for reliable transportation rather than building one. Build what is unique to your business, and buy the components everyone running AI needs.

Least Cost Routing, Borrowed From Telecom

For anyone who came up in telecom, least cost routing is a familiar concept applied to a new problem. A model router evaluates each request for complexity and subject, then selects a model against capability, availability, and underlying token cost. Image generation, deep research, and a weather lookup should not all reach for the same model.

Routing also carries access control. Smith’s example: a marketing team routed to a model strong at image generation and creative work, while developers route to a privately hosted model that excels at code at a tenth of the cost. Reporting on override behavior turns this into enablement, showing where one user’s habits diverge from peers who are saving substantial token volume by letting the router work.

The related trend he is seeing fastest is not organizations training their own models but running privately hosted open source models for specific jobs: call transcription, coding, mid-level summarization that does not require complex reasoning. A service provider can multi-tenant a privately hosted model across customers without combining their data, which matters because owning the hardware outright generally requires sustained utilization above seventy percent to beat public API pricing.

On lock-in, Smith’s warning is about ecosystem lock-in rather than vendor lock-in, since you will pick a vendor eventually. Committing entirely to a single model ecosystem means accepting its pricing and its release cadence, and it removes cost management as a lever, because model prices change weekly.

Forecast Total Cost of Ownership Before You Forecast ROI

Observability and ROI tracking tend to come last, after the project is built. Smith’s reframing of the approval question: organizations ask whether they can build something and how, and rarely whether they should.

What he recommends green-lighting against:

  • Data that is already available and in clean condition
  • High-volume transactions with a known value per transaction
  • The full workflow cost, including external API calls, external tooling, and any human steps, not just the model cost
  • A clear comparison against how the work is done today

On pilots, his position is that a pilot built purely to create familiarity with the tooling does not need to show ROI, but it needs a short, defined window. Most pilots he designs aim for the first project to pay for the platform. He also revisited the widely cited MIT study on AI pilot failure, noting that the five percent success figure reflected self-built efforts while partner-led implementations in the same data reached around sixty percent.

For more on the gap between AI investment and realized value, see Why Do So Many AI Investments Fail to Deliver Business Value? and What Should You Look For in AI Governance Tools?.

AI and Infrastructure Are One Decision

The thread connecting both halves of this conversation: AI decisions and infrastructure decisions pre-make each other.

Commit to all hyperscale cloud and all public models, and you have constrained your options. Refresh all your hardware and keep it on premises, and latency and paying twice for data movement become live questions. Smith attributes the resurgence in private cloud to organizations recognizing that keeping one copy of data adjacent to where AI processes it, and paying for a security layer once, changes the economics. Software has become cheap enough to build that organizations are writing their own again, and that software needs somewhere to run.

Which is why the VMware Cloud Foundation deadline and your AI roadmap belong in the same budget conversation. An unplanned migration can absorb the capacity and budget you set aside for AI. We cover that in our companion post on what the October 2027 VMware Cloud Foundation deadline means for your compliance.

How Amplix Can Help

Amplix is vendor-neutral and cost-free to engage, which matters when the question is whether to buy an AI control platform, self-host models, shift to CapEx, or keep consuming public APIs with better controls. We help you scope total cost of ownership before the project is approved, evaluate providers against your data and compliance requirements, and make sure your AI and infrastructure decisions are made together.

Review our AI capabilities or our technology strategy services, or talk to an Amplix expert about getting AI spend under control.


Frequently Asked Questions

What is AI cost management?

The practice of forecasting, monitoring, and controlling AI consumption spend across models, agents, and infrastructure. Unlike software license management, it has to account for variable usage, since cost scales with consumption rather than seat count.

What is shadow AI?

AI tools and platforms employees adopt without IT approval, including personal subscriptions and applications built on unvetted development platforms. It is typically a response to the organization not providing adequate sanctioned tooling, and it makes consumption and risk invisible to IT.

How can we reduce AI costs without reducing output quality?

Prompt caching on input tokens can cut consumption by up to ninety percent. Batch processing for non-real-time work runs about fifty percent cheaper on public models. Routing each task to the least expensive model capable of handling it often produces the largest savings, since defaults favor the most expensive option.

What is least cost model routing?

A routing layer that evaluates each request’s complexity and subject, then selects a model based on capability, availability, and token cost. It can also enforce role-based access, so different teams reach different models by default.

Why do AI pilots fail to show ROI at scale?

Pilot forecasts usually capture build cost only. Running at production scale requires hosting, security, observability, and reliability infrastructure that was never priced. The MIT study often cited on pilot failure showed a large gap between self-built and partner-led outcomes.

Should we self-host AI models or use public APIs?

It depends on utilization and use case. Privately hosted open source models are cost-effective for defined, repeatable tasks. Buying your own hardware generally requires sustained utilization above seventy percent to beat public API pricing, though a service provider can multi-tenant a hosted model to improve those economics.


Learn more about our content creation process

Share this:

Key Takeaways:

  • AI budgets break because organizations price a license while buying consumption-based infrastructure.
  • Most organizations are in a build phase but budgeting for a run phase, and developer usage can run twenty to a hundred times an average employee’s.
  • Volume growth, not per-token pricing, is the dominant cost driver, though hardware costs and the end of subsidized flat rates add real pressure.
  • Prompt caching, batch processing, and right-sized model selection are the highest-leverage reductions available.
  • Dashboards report spend after the fact. Control requires enforceable limits, throttling, automatic model fallback, and step-level agent replay.
  • Treat agents as digital employees requiring management and oversight rather than tools you switch on.
  • Total cost of ownership has to be modeled before ROI can be forecast meaningfully.
  • AI and infrastructure are a single decision, which is why the VMware deadline and your AI roadmap share a budget.
Share this:

Insights in Your Inbox

Never miss what’s new from Amplix! Subscribe to get notified.

Ready to amplify your technology investment?