What Databricks’ AI Spend Can Teach the Rest of Us

As AI agents and tools become part of our clients’ day-to-day operations, managing AI spend has become a bigger topic of conversation for our consulting teams. In this article, we cover five lessons lessons from Databricks’ own experience managing AI spend, including why the most powerful model isn’t always the right one, how visibility can change user behavior, and why agent behavior may be one of the easiest places to miss unnecessary spend.

And the concern is understandable. AI introduces new variables into technology spending: different models, reasoning levels, token consumption, agent behavior, tool calls and autonomous workflows that can keep spending long after a user hits “go.”

Databricks recently shared some of what it has learned from managing this problem internally in its Govern AI Spend at Scale webinar. Their AI budget may be bigger than most, but the lessons are practical regardless of your token spend, model, or AI stack.

The overarching lesson: AI cost management is becoming less about what a model costs and more about how efficiently people and agents use it.

Lesson #1: Not every task deserves your most powerful model.

This was one of the most concrete lessons from Databricks’ own experiments.

As coding agents became more widely used internally, Databricks benchmarked them against real development tasks. Rather than simply comparing model prices, the team looked at both the success rate and cost of completing those tasks under different configurations.

One of the things they tested was reasoning effort. The assumption might be that giving a model maximum reasoning capability will produce the best result. But Databricks found cases where dialing reasoning effort down significantly reduced cost without materially affecting success rates.

They saw a similar pattern when comparing different model and agent combinations: more expensive wasn’t automatically better for every task.

That changes the cost conversation. The question isn’t simply, “Which model is best?” It’s, “What level of capability does this particular task actually require?”

A simple task may be perfectly well served by a smaller, cheaper model or lower reasoning setting. A genuinely difficult problem may warrant the additional horsepower. Once you understand the difference, you can even begin routing tasks accordingly instead of asking every user to make that decision.

The lesson: Benchmark models and configurations against your own workloads. Measure cost alongside quality, and reserve your most expensive resources for the work that actually benefits from them.

Lesson #2: Make AI spend visible to the people creating it.

Databricks encouraged its engineers to experiment with coding agents, but as adoption grew, so did costs. Engineers could choose powerful models and high reasoning settings, and there wasn’t much incentive for an individual user to think about what those choices cost.

One of Databricks’ responses was surprisingly simple: make that consumption visible.

Instead of AI spend disappearing into a centralized bill, users could see what they were consuming and how they were tracking against budgets. Once engineers could see what their choices were costing, they had an incentive to use AI more thoughtfully.

That’s an important distinction. Central visibility is useful for the people responsible for the overall AI budget. But it doesn’t necessarily change the decisions being made by the people actually consuming those resources.

Put the information closer to the decision, and behavior can change.

The lesson: Give users and teams visibility into their own AI spend and some ownership over managing it. They should know what they’re consuming, what “normal” looks like, and when they’re approaching a budget.

Lesson #3: Optimize the task, not just the token.

This one is easy to miss.

When AI costs climb, don’t just look at the model. Look at agent behavior, too.

Databricks analyzed agent traces and found inefficiencies that had little to do with the underlying price of the model. One example was repeatedly sending the same repository context without caching it. Every unnecessary repetition added cost without adding value.

A more dramatic example: Databricks described an agent that would typically spend around $50 completing a task. During one debugging session, it decided to read through a huge collection of log files. The task ultimately ran up thousands of dollars.

The model didn’t suddenly become more expensive. The behavior of the agent changed.

As agents become more autonomous, there are plenty of places for this kind of waste to creep in: repeated context, excessive reasoning, retries, unnecessary tool calls, loops, or simply taking far more steps than a task warrants.

That’s why token consumption by itself only tells part of the story.

A more useful question is: How much does it cost us to successfully complete this task?

That shifts the unit of AI economics from the individual token or model call toward the outcome you’re actually trying to produce.

The lesson: Trace expensive workflows and understand what agents are actually doing. Cost per successful outcome can tell you much more than token price alone.

Lesson #4: Put guardrails around runaway spend.

The $50-to-thousands example points to another challenge that becomes more important as organizations adopt agents: the person initiating a task is no longer making every spending decision.

A user can give an agent a goal and walk away. From there, the agent may decide how much context to retrieve, which tools to call, how many times to retry, and how long to keep working.

That autonomy is useful, but it also introduces financial risk.

Waiting until the end of the month to discover an unexpectedly expensive workflow isn’t much of a control. Organizations need ways to identify abnormal consumption while it’s happening.

That could mean budgets or thresholds at the user, team, agent, workflow, or task level. And reaching a threshold doesn’t necessarily need to mean an automatic “no.” A particularly valuable task may legitimately need additional resources.

The important thing is that unusually expensive behavior becomes visible and intentional rather than continuing indefinitely in the background.

The lesson: Put limits around autonomous spend, with a path to approve more when there’s a legitimate reason. The goal of a guardrail isn’t to stop valuable AI work. It’s to make unusually expensive behavior intentional.

5. Treat AI cost management as a feedback loop.

The examples above all point back to a simple framework Databricks used throughout the webinar: see it, show it, act on it.

• First, see it. Understand where AI spend is actually coming from. Which users? Which models? Which agents? Which tasks? What happened inside an unusually expensive workflow?
• Then, show it. Put that information in front of the people who can do something about it, including the teams and individuals creating the spend.
• Finally, act on it. Change model configurations. Adjust reasoning levels. Fix inefficient agent behavior. Route simpler work differently. Set budgets and guardrails where the data tells you they’re needed.
• And then keep measuring.

That last part matters because AI usage isn’t static. New models appear. Prices change. Agents take on new tasks. What was the most economical configuration six months ago may not be the right answer today.

The lesson: Don’t treat AI cost governance as a one-time exercise where you choose models, set limits, and walk away. Use what you learn from actual usage to continuously refine how AI gets used.

Questions to ask to keep AI spend under control

It’s tempting to think about AI cost management primarily as a pricing exercise: How much are our tokens? Which model is cheaper? Can we negotiate a better rate?

Those questions matter. But Databricks’ internal experience points to a much bigger set of levers.

• Which model are you using?
• How much reasoning does the task need?
• Can the person creating the spend see it?
• Is the agent working efficiently?
• What happens when its behavior suddenly changes?

As AI moves from individual prompts to autonomous workflows, those questions will only become more important.

The organizations that get AI costs under control won’t necessarily be the ones that use the cheapest models. They’ll be the ones that understand what’s driving the spend well enough to optimize it.