Plan for midstream adjustments

When we launched the first iteration of Lilli, McKinsey’s internal gen AI knowledge tool, it caused a surge of excitement. Here we were, using a revolutionary capability to create something special. Interest grew, and our people threw themselves into using it.

As dozens of new use cases were piloted on the platform, the team realized, however, that the platform was straining. Lilli was built on a number of large language models (LLMs) with a single provider, and costs were rising fast as more people used more applications. The team did some further analysis based now on real usage patterns and could see how different LLMs performed differently in the trade-off between speed, accuracy, and cost.

That analysis confirmed a critical insight: A one-size-fits-all approach was creating a value bottleneck. Relying on a single provider meant that the Lilli development team couldn’t optimize for the specific needs of each use case. Some applications required maximum accuracy, while others prioritized speed and cost efficiency. We had hit a scaling wall, not just in terms of cost, but in our ability to deliver the right capability for the right job.

So the decision was made to pivot and rearchitect Lilli around a composable open-source platform that was LLM agnostic. That decision was critical in enabling the platform to scale economically.

Each company will run into unique issues along its tech and AI transformation journey. That’s normal, but you need to be ready to adapt. In our experience, there is a common set of issues that companies struggle with. Here are some of the most important ones, and how to solve them.

Road map and value challenges

The issue: Getting domain leaders to commit.

Often, domain leaders are reluctant to commit to a domain transformation because it is thrust on them and includes incremental productivity or growth targets beyond their own existing commitments. This is especially true when the costs of the implementation are being allocated to their P&Ls. If the organization has traditionally struggled to execute technology-led projects well and on time, the reluctance increases.

The solution: Give them ownership and smooth the runway.

Prioritizing domains and putting domain owners in place who will actually roll up their sleeves to create the road map and do the work diminishes the leadership commitment issue a lot. Look for ways to reduce the risk to the domain.

A beverage company, for example, covered the first couple of quarters of the transformation implementation costs for a targeted domain centrally. While the organization tracked objectives and key results (OKRs) rigorously, it did not initially increase the target for resulting revenue growth for the first year.

The issue: Scaling costs start to spiral.

Early in the transformation, cost estimates are rough. That’s generally OK because they are accurate enough to provide internal and external stakeholders with some predictability. But there is a lot you don’t know until you actually start building and implementing solutions, especially as the AI solution and pricing ecosystem evolve rapidly.

The solution: Go deep on the unit economics of scale.

It’s important to revisit your original cost estimates, but the real key is getting a handle on the unit economics of scale (meaning those elements that specifically impact costs related to scaling).

Here are a few important elements to focus on:

• Replication. Inefficiency sets in when each new business unit or market requires heavy rework. Variations in data schemas, process flows, local regulations, and user interfaces often require heavy rework.
• Run cost. Unforeseen AI costs often emerge, especially as solutions move from simple prompt-response models to more agentic architectures. As agents involve multiple model calls, trigger external tools, and run retrieval steps, usage-based fees can spike far beyond initial forecasts—and that’s not even to mention some of the additional hidden costs like data movement and vector storage. Teams should forecast and track cost at the unit level (cost per workflow, or per agent task, for example) and value level (average cost per customer ticket closed or sales proposal generated, for example) and not by token or in aggregated total spend. Also, carefully consider the architecture of the solution, which can have a 10–100 times effect on reducing costs.
• Maintenance and drift. As AI systems scale, maintaining performance consistency becomes a hidden but significant cost driver. Models degrade over time as underlying data distributions shift—what works in one quarter or business unit may underperform in another. To counteract this “model drift,” teams must retrain, revalidate, and redeploy regularly, requiring dedicated machine learning operations (MLOps) infrastructure and staff.

As you come to grips with the unit economics of scale, think through how scale is possible without adding more headcount, especially through automation and agentic solutions, while being clear about the critical role humans will continue to play.

The issue: Leadership doesn’t have reimagination abilities.

Senior business leaders are consumed by their day-to-day work or have evolved as excellent people managers, but their creative business “reimagination” skills may have atrophied or perhaps were never their strength to start with.

The solution: Get entrepreneurial domain owners.

The answer here is to find domain owners and product owners who have a strong entrepreneurial streak (not just a glorified manager) and providing them with sufficient support for them to exercise it. We’ve found that developing a product management chapter in which all business product owners across domains enroll can be an important way to keep that entrepreneurial muscle alive. Toyota and Freeport-McMoRan, in particular, made this entrepreneurial trait a focus of their team building for their AI transformation programs.

Talent challenges

The issue: A “two-tier” talent model.

An “us versus them” culture can take hold if you’re not careful, with the “us” teams working on the cool, high-profile AI solutions and other teams doing the less engaging work, like maintaining core systems. Compensation and growth path differences will exacerbate this issue.

The solution: Have a clear “talent migration plan.”

Ambiguity with a two-tier model is inevitable, but the key point is that you can’t let it ossify and become the norm. Create clear delineations between job descriptions, skills expectations for each job, and the associated rewards. Establish an upskilling program that allows traditional technology talent with the aptitude to cross over into tech and AI transformation work.

At a consumer company that established an AI factory separate from the IT organization, internal candidates went through the same coding tests and interview process as external ones. About 20 percent of them passed and became part of the AI factory.

Another point to remember: Your core systems don’t need to be “free” from innovation. There is plenty of room to innovate here and create opportunities for that talent to stay at the edge of their craft.

The issue: Keeping your top developers.

Rewired, Second Edition

This updated edition offers brand-new insights into cutting-edge AI solutions—and what it takes to implement them—as well as the new economics of digital and AI transformations.

Learn more

Maintenance is crucial to ensuring scale, but many of the top tech people who developed a solution aren’t interested in the less glamorous work of supporting it. Either they leave, which undermines future development, or maintenance work becomes a low priority for them.

The solution: Live by the core principle of leading tech companies: “You build it, you run it.”

Maintenance is part of every developer’s job. This task should get easier as gen AI tools are providing increasing leverage for legacy maintenance tasks. Be clear about which utility systems should be outsourced along with their maintenance.

The issue: Not enough team owners.

Having a sufficient number of qualified cross-functional team owners (sometimes called owners) is almost always an enormous challenge. Companies may be able to find a handful to support the first couple of domains, but then run out. Tapping project managers to assume the role often doesn’t work out.

The solution: Make the team owner the “manager of the future.”

Keep their visibility and recognition high, while reinforcing the importance of the role in the business’s future (and in a person’s growth). Ensure function leaders make the work of team owners a standing agenda item in leadership meetings and promote the best performers to become members of the function leadership team.

Operating model challenges

The issue: Sustaining an AI factory separate from core IT.

Over time it can be challenging to delineate the responsibilities of the AI factory producing AI solutions versus the traditional IT organization. People can become confused about who does what, and where they should go to get the technology services they need. This confusion can gum up the works and slow solution development progress.

The solution: Clearly articulate roles and responsibilities.

Clarify responsibilities top down. For example, the AI factory will be responsible for all new domain reimagination and any resulting digital and AI builds. IT leaders will support security, data access, and cloud compute, and take lead delivery responsibility for core system modernization or integrating prebuilt software packages.

Establish handoff points of contact and liaisons so that digital and AI solutions flow to the right units across the business. A common governance structure for funding based on business cases can also help settle issues. In essence, you need to take the time with senior leadership to clarify the operating model blueprint.

The issue: Having an operating model just for IT.

The CIO is on board with implementing a new operating model but feels it’s too hard to align with other leaders on it. It leads to an “agile in IT” world, which can help with IT projects but not tech and AI transformations.

The solution: Get the CEO to intervene.

We don’t know of a single effective operating model implementation where the CEO was not involved in aligning the top team on the target blueprint. If the CEO isn’t convinced and committed, your chances of success are basically zero.

Data challenges

The issue: Sustaining data product orientation.

Many companies start their transformation with great intentions to make data an accelerator. They establish a data product architecture and a clear view of how these products can support numerous future domains. But over time, the data products developed for the initial domains atrophy as the budget is allocated for new builds, and data owners and engineers move on to other priorities.

The solution: Align incentives to ongoing support.

Align incentives so that maintenance and scale costs are appropriately split between data owners and business users. Consider adding KPIs, such as percentage of data reuse, for solution teams to reward the development of capabilities that enable scale. Some companies have created internal charging systems to incentivize maintenance-friendly behaviors.

The issue: Data ownership jealousy.

Business units and business functions often want to “own” their data because data is power and controlling it limits dependencies. This leads to a proliferation of data sources and platforms and decision rights on access. The result is labyrinthine and time-consuming journeys to find and access the necessary data to develop models.

The solution: Align on clear decision rights.

These need to be set up front. If you can negotiate it with relevant leaders, so much the better, but more often than not, it requires an edict from the CEO to share data.

Tech challenges

The issue: Balancing IT modernization with the AI agenda.

AI builds pay off in quarters, whereas for core systems modernization, it’s often in years. Yet they compete for the same resources, both financial and people. This undermines smart allocation decisions.

The solution: Tech and AI investments need to stand on their own merits.

Any investment in technology, whether that’s a new AI solution for the transformation or an enterprise resource planning (ERP) upgrade, needs to have a clear business case attached to it. This clarity across AI transformation and tech modernization initiatives is the only route to better allocation decisions.

With this clarity ensconced, it’s important to define clearly how core modernization and AI advancement are mutually reinforcing priorities within one business transformation plan. The focus should be on modernizing data and integration layers first to get at the “low-hanging fruit” opportunities, and adopting modular architectures that let legacy and AI systems coexist.

Change management challenge

The issue: The tech and AI transformation falls off of the top agenda.

We often see the temptation for a tech and AI transformation to be displaced from a CEO’s “top three” priorities. This tends to happen when the top team does not have sufficient conviction that the domain road map will generate game-changing value.

The solution: Be intentional about regularly realigning on tech and AI transformation goals.

In practice, it’s hard to maintain momentum over the long term, especially if you did not build conviction during the domain reimagination phase. If that’s the case, call a time-out. The CEO and C-suite should take two to three months to realign on goals and rebuild conviction based on compelling business cases. Be ruthless. If you can’t find a compelling case, call it out and move on to something else where there is real payoff potential.

The critical point here is that a transformation road map is a living document that needs to be reviewed and refreshed. Committing to a tech and AI transformation is really about recommitting to it regularly.

Having sufficient enthusiasm and focus at the beginning of a tech and AI transformation is rarely an issue. By year two, it often is. Small problems can grow into large ones, inevitable setbacks drain energy, and fracturing alignment saps momentum. Most companies we’ve profiled in this book have faced these kinds of issues. Their leadership then stacked hands and made the necessary midstream adjustments.

When a tech and AI transformation takes off, it creates a positive flywheel effect. But it needs continuous focus, conviction, and investments of leadership energy—particularly from the CEO.