Lucia Rahilly: We’ve reached an interesting juncture. AI use is becoming mainstream, but enterprise value, not so much. What does the research tell us about what separates organizations that are simply using more AI from those that are beginning to change the economics of their business? Tanguy, let’s start with you.
Tanguy Catlin: The latest State of AI research we published shows adoption is already strong: enterprises we surveyed who say they’ve adopted AI have grown from 88 to 89 percent. What’s more interesting is that while AI adoption is broadening to more functions, the real signal is that more companies are now scaling AI. We were just shy of 40 percent. Now, more than 40 percent of respondents say they are truly scaling AI.
And yet you ask: “Where is the value?” Most respondents, 80 percent, say their individual productivity has increased. Yet only 6 percent can show an impact on EBIT [earnings before interest and taxes] of 5 percent or more. Let’s call them AI high performers.
The real question is no longer adoption; it’s value creation. What can we learn from those high performers? It’s not simply that they use more AI—though they do. But they’re focusing on AI where it matters: on the areas in their industry that create operating leverage for value creation.
Critically, they are not deploying AI on top of existing workflows and processes. They pause and do the hard work of reimagining and redesigning end-to-end workflows around AI. While most organizations are obsessed with productivity and efficiency, as we’ve observed in our survey, the winners tend to have an agenda focused on growth and innovation, alongside efficiency.
They also have stronger leadership and strong measurement discipline around deployment. In short, the focus is no longer on where to deploy AI but on changing the economics of business through AI.
The focus is no longer on where to deploy AI but on changing the economics of business through AI.
Lucia Rahilly: As AI investment rises and it shifts from pilots to day-to-day operations, many leaders have discovered that AI spend behaves very differently from a traditional software budget. Where does the economics problem tend to show up first, and what catches organizations by surprise as they scale?
Tanguy Catlin: Let’s anchor ourselves in facts. Today, more than a quarter of organizations report spending more than 10 percent of their total technology and communications budget on AI. The spend is becoming material and visible. It’s also growing fast: 60 percent of respondents plan to keep increasing their AI spend next year. But that’s starting to create a challenge, as you mentioned.
One in five respondents, about 20 percent, say AI-related operating costs are starting to constrain their AI use, requiring them to take action. One of the drivers is token costs.
Not surprisingly, this tension is most visible among the high performers. They were the first to invest, and they tend to invest more than their peers. One of the areas they focused on was the software development life cycle, where agents are used to automate coding, which is very token hungry.
They were the first organizations to report the challenge of managing AI spend, and that’s why you can learn from them on governance. They were the first to create visibility into what’s being consumed, by whom, and in which workflows. More important, they started measuring the business outcomes from token consumption and began optimizing.
Lari Hämäläinen: It’s worth saying that when we look at adoption this year, we’re still at an early stage of what we would classify as enterprise AI. To give a concrete example, while Tanguy said that nearly 90 percent of organizations use AI in at least one function, most of that usage is still personal tooling: chatbots for individuals or co-work-type tools, and even tools like coding agents. In our data, only about a third of large companies have scaled them across the organization. For all the talk today about AI usage and its consumption, there’s still a lot of runway in software development if two-thirds of organizations have not yet scaled those tools.
We’re only starting to address true enterprise workloads: the processes a company runs, from back office to commercial to supply chain. Most AI usage so far has been individual-tools-driven, which also enabled it to scale fast.
We’re only starting to address true enterprise workloads: the processes a company runs, from back office to commercial to supply chain. Most AI usage so far has been individual-tools-driven.
On your point about AI high performers, the 6 percent: I was surprised by this year’s survey results because the distinction in how these organizations apply AI is dramatic. Most of the survey population does not go beyond personal tools.
Comparatively, the 6 percent who feel they’re getting results, or what some call “AI-native development,” are reengineering their businesses, rebuilding processes and their supporting organization around AI.
We’re now seeing this in traditional processes but also in software development. For example, the current direction is not necessarily to change all teams, but to enable them with tools, then rebuild products and product modules from scratch with AI, which can get you higher productivity and speed.
Tanguy Catlin: I want to emphasize the most important thing people need to realize. This is not the first time we have had a technology that can transform every industry and function. We call these general-purpose technologies. We’ve seen them with electricity, steam, et cetera.
Robert Solow, the Nobel Prize-winning economist, observed this paradox: It takes a long time from the moment a new technology is introduced to the moment its impact can be measured across society. The reason is exactly what you emphasized. You need to adjust and adapt to capture the value from the technology.
We see too many of our respondents making the mistake of deploying technology on top of existing processes, and they probably get 20 cents on the dollar. By contrast, it is the winners Lari described who are reimagining entire workflows around AI. In the area of software development life cycle, automating code generation is not the biggest problem; you need to automate everything from requirements gathering through testing; otherwise, you get very little throughput into production. The important takeaway is this is about reimagination.
You need to automate everything from requirements gathering through testing; otherwise, you get very little throughput into production. The important takeaway is this is about reimagination.
Lucia Rahilly: There’s a paradox at the heart of this conversation. We’ve spent years hearing that AI is getting cheaper, yet many enterprise bills are moving in the other direction, as you’ve described. If AI models keep getting more efficient, why isn’t the enterprise AI bill trending down? What’s driving this disconnect between cheaper intelligence and higher total spend?
Lari Hämäläinen: The first thing to say is that intelligence at a certain capability level is getting more affordable. If we baseline to GPT-3.5-level capability, which came out in early 2023, it’s estimated that the capability level today can be delivered by models with inference more than 280 times cheaper. Instead of $20 per million tokens, you’re now paying only a few cents. We’re seeing a similar phenomenon: once a capability level is reached, after some time, we’re able to deliver efficient results with more efficient models.
That still holds true. But there are a couple of interesting considerations. First, most model usage comes from the commercial labs: OpenAI, Anthropic, Google, and so on. These labs offer only a few of their latest models to end users. The capability of these models relative to the price you pay has increased dramatically. That’s worth reflecting on. When you look at LLM [large language model] pricing overall, there’s a bump each time a new model family arrives. This bump comes because when models are bigger, they cost more to inference. Over time, the models get optimized, so you can run the same model, or a distilled version—a smaller model with a similar capability set—more cheaply. So, we see LLM pricing fluctuate.
The second thing to acknowledge, especially with the commercial models, is that in the past year, gross margins for inference have nearly doubled. This means companies have held prices up significantly to double the margins compared with the falling cost of inference. We’ve moved from about 35 to 40 percent to the mid-60s for gross profit per inference, and above 80 percent for some API businesses. The reason is that labs were making tremendous losses, even on inference, and as they prepare to become commercially viable companies, and maybe pursue IPO, they have to fix their financials.
As for the token usage you referred to, the simple way to think about it is what we use in chat. Everybody uses chat-based tools or, at most, an image generator. Now we’re using agentic tools much more, though still as personal tools, such as Claude Cowork, which can work autonomously and use thinking models. To give an example, in thinking models, the thought process and output can result in token usage of five to ten times, and far more on hard problems.
We then apply that to our work, especially in software development, which involves running swarms of agents that independently manage large code bases, which has driven up token usage. But there are also side effects. As token usage for software development increased, we find agentic coding writes roughly eight times more code than humans do. Where a human might touch 10 percent of the code, an agent might rewrite the whole code base in one go, every time. So, there’s a lot to learn about working effectively with these tools.
Lucia Rahilly: Let’s get into the difference between working with chat-based AI, LLMs, prompts, responses, the fancy footwork agents do, and how that difference changes the economic calculus. Two runs of what looks like the same task can cost very different amounts. Tell us about that.
Lari Hämäläinen: Only this year we’ve started to understand how these agentic systems behave, and they almost have their own laws of nature. It’s fascinating. An example of a law is that when you give an agent an identical task, the output cost arises at distribution. It’s a probabilistic distribution because the agent can take different paths to solve it, especially for a longer task. We find that the variance can be up to 30 times between different runs.
Imagine you’re running an operation, let’s say the sales and operations planning process for your supply chain, and every day there’s a 30-fold difference in cost. Agentic processes are highly variable in cost. Second, when you look at what’s driving the cost, it’s not just the output. A lot of the cost is in refining the output, especially when agents take probabilistic paths, and that’s where the big price differences come from. Finally, system engineering becomes like a core skill. With traditional software development, we worked with deterministic software, and many times we weren’t thinking about things like efficiency.
Now with agents, how you solve a task matters tremendously. For example, do you use a single agent or a multiagent swarm? Even how you call a function matters: Do you use MCP [model context protocol], an agentic protocol, or a direct API [application programming interface] call?
That choice can have a cost difference of roughly ten to 20 times for the agent. We’re only learning much of this now. Most enterprises haven’t had to face this before, as we haven’t addressed operational processes yet, but in the next 12 months, it will become relevant.
Lucia Rahilly: This also raises a measurement problem that one of our audience members asked about in the preregistration questions. In agentic workflows, the model call can be only part of the fully loaded cost once human review, exceptions, and errors are included. If cost per token is the wrong unit of value, what should leaders measure to decide whether an agent is worth it?
Lari Hämäläinen: I believe you should look at operational metrics around cost per task execution and quality rates; these are metrics we already use in most of our processes to judge whether to move to agentic technologies.
Imagine it like this: If coding something takes you an hour but verifying the code takes six minutes, it’s probably worth having an agent do the task, as long as its success rate is meaningfully above 10 percent, because six minutes divided by an hour is 10 percent. You then gain time back and free up capacity for other things.
That’s a simplification, but it’s an easy way to start thinking about two things. First, agentic technologies apply to a lot of repetitive processes, because their probability of success is much higher. And it comes back to the question we discussed earlier: How do I reengineer the rest of the work around it?
Tanguy Catlin: I would add that there is sometimes a hidden cost. If you let the model do something and it makes mistakes, sometimes the cost of recovering from those mistakes can be extremely high. We’ve often seen wrong trade-offs in the selection of cheaper models because organizations want to optimize costs without realizing that a higher-quality model often more than justifies its higher cost.
We’ve often seen wrong trade-offs in the selection of cheaper models because organizations want to optimize costs without realizing that a higher-quality model often more than justifies its higher cost.
Lucia Rahilly: We’ve had a lot of questions from our audience about how to optimize AI spend. Once leaders can see the economics, the instinct may be to grab the obvious levers you just mentioned: cheaper models, tighter limits, and fewer tokens. Where do the biggest savings come from, and how do you optimize spend without optimizing away the value?
Tanguy Catlin: It’s a brilliant question, and I wish there were a simple answer. The truth is, there is no single cost lever. When we support our clients, we optimize across a portfolio of about 40 levers spanning forecasting, optimization, sourcing, and governance. But I’ll try to break it down into three parts. The first one is taken for granted, but it’s extremely hard: creating visibility and attribution, which means understanding which use case, business unit, agent, model, or user is driving the spend.
There are tools, such as AI gateways, model registries, spend dashboards, and anomaly detection, that can help make the economics observable. But in our experience, it is not a simple task, and it’s critical. Once you have that transparency, you can move to the next level, trying to forecast where your spend will come from.
That’s the first layer: visibility and attribution. The second layer is technical work. This is where you need to do workflow optimization, matching model tier to task complexity. Spoiler alert: most organizations are using advanced models where they’re not necessary.
You want to route requests dynamically and cache reusable context; Lari described how models consume context. You want to teach your organization to reduce unnecessary prompts and tool payloads, and to put limits on loops and tool calls.
There’s a massive amount of work on the technical side and on workflow optimization. The third layer is demand and sourcing discipline. You’re buying tokens from many sources, so can you remove unused licenses? Can you manage quotas and exceptions? Can you improve utilization?
How do you negotiate provider terms in this dynamic environment? How do you keep options open across model types and providers, so you don’t lock yourself in? I would emphasize that sourcing discipline is the third dimension. We’ve done this within the firm. We’ve achieved impact north of 20 percent, and, in several areas, 40 to 45 percent. The impact is very significant, but it requires true FinOps [financial operations] discipline.
I want to call out one thing that I think was implicit in your question. The danger is approaching this as a cost-cutting exercise. The goal should not be indiscriminate cost cutting. What you want to do is define where spending on intelligence delivers high returns for your organization and how to optimize for that return.
The danger is approaching this as a cost-cutting exercise. The goal should not be indiscriminate cost cutting.
Lucia Rahilly: As you’re saying, cost is clearly important, but no company wins by becoming the world’s best token saver. What has to change in the operating model for AI to move from a productivity tool to a genuinely agentic way of working? And where do the bigger gains come from?
Tanguy Catlin: First, if you just deploy AI as a tool, like chatbots, you’ll get very little. You’ll improve employees’ competency, and it’s good for literacy, but you’ll see few performance improvements. The impact starts when you focus on the workflows that are critical to your organization’s operating performance and redesign them end to end, and understand how AI will unlock value.
Now we’re entering a third stage beyond redesigning workflows; we are embedding agentic workflows. The next frontier, I think, will be agent factories, in which people learn to supervise much larger virtual workforces of agents. That is the next massive unlock we are working toward.
Lucia Rahilly: Let’s get super practical. Lari, if I’m a CEO, what decisions do I need to make now to ensure AI becomes a scalable source of value rather than just a growing cost base?
Lari Hämäläinen: A few things. I think every CEO should be running two or three lighthouses in their company. The lighthouse is an area of a business that matters and which you’ll rebuild with AI. If you believe AI brings the ability to do things differently and it’s a fundamental technology, then choose two or three areas and start experimenting.
Don’t just give your people tools; rebuild those areas. Almost skunkworks it, to use an old word. Second, what will matter a lot in these areas and beyond is owning your data. We’re waking up to a reality where all data matters.
That includes contextual data: meeting notes and transcripts, or any documentation of your processes. All of it is extremely valuable, especially as agents become part of our processes. Third, I’m a big believer that a huge capability shift is occurring for every enterprise.
Too many companies, to my taste, still outsource much of their technology talent, and they run processes on big software platforms that will become core to their future. The question to consider is what capabilities you want to own and bring in-house, and how you’ll build the muscle to run your organization with them and integrate them.
Imagine that your organization is 100 percent people today, but three years from now it could be 50 percent agents and 50 percent people. The ability to work in that hybrid model will be critical, and none of us wants to outsource it. That brings me to point four: I’m seeing a huge shift in what’s called the boundary of the firm.
That simply means what we do ourselves, what we outsource, and where we do the things we do ourselves. Finally, AI costs need to be managed very tightly.
