After many years running a large data business built on top of artificial intelligence models, I’ve seen firsthand how AI too often becomes a solution looking for a problem. When any organization’s expectation is for a significant amount of impact thanks to data and AI, the reality – which is that AI’s impact is often pretty vague – can create uncomfortable situations. 

Of course, data leaders should be used to this dynamic, because data has been treated in the same way for the past 10 years. Just improve our aptitude with data, the thinking runs, and our company’s performance will be transformed. 

Neither scenario is necessarily true, and certainly not that straightforward. 

My point is that there are many places inside your business – customer-facing products, for example, or operations – where it probably makes sense to leverage AI and where you could realize significant ROI. But you’ve got to go about it in the right way to see those returns. Here are a few strategies I’d recommend to any data leader grappling with the topic of AI, data, and business impact.

  1. Partner with the business. Before the general idea of AI leading directly to ROI triggers some highly specific investment decisions you may regret, reach out and partner with the businesspeople in your company. Come up with a list of what these users know about the capabilities of AI, and where they think it can improve operations. You’ll quickly discover that people have widely differing ideas about where they think data and artificial intelligence could have the most impact as well as the problems they’d like to address with it.  
     
  2. Separate the AI wheat from the chaff. If you have any familiarity with AI you’ll see right away that some ideas from the business will be clear winners. The use cases are great, and they may solve significant problems. For example: there are certain customer requests you can automatically fulfill because you can get a better understanding of what they’re asking for with AI, or there are certain data deliveries that people can’t quite execute manually, but are now within reach because of large language models (LLMs). You’ll also see doubtful cases just as quickly: marginal problems that will barely work from a technology perspective, or where data density is too low for effective automation. So move forward with the few, top ideas that you feel highly confident about based on their potential ROI. If you staff for 50 ideas but only deliver two, it’s going to be a big problem for you.
     
  3. Define what true impact means. A lens I always use to judge an AI application in a business is to ask, “What’s the worst thing that could possibly happen in this use case?” Machine learning success rates of 95 or 97% might represent an amazing improvement over a manual process, but could be disastrous if the application is a self-driving car. (Imagine experiencing a car accident 3-5% of the time when you take 700 trips a year and you’ll understand what I mean.) The number one rule of machine learning at a forward-thinking AI company like Google is don’t use machine learning if you don’t have to. If it’s not imperative to use AI to accomplish a specific business goal, then you don’t have an AI use case.
     
  4. Keep an open dialogue. A day hardly goes by when you don’t read something about AI replacing jobs. While it’s hardly credible to predict that AI won’t supplant some highly repetitive or highly structured jobs, I think it’s worth pointing out to anxious colleagues that general technology progress and US offshoring probably have already replaced many of the highly repetitive non-physical jobs in our economy. The airport is a great example. Thanks to existing technology you check in for your flight at a standalone kiosk and order food from a screen. Convenience also plays a role in coding. Why write out more than a dozen lines to determine whether the sentiment of a text is positive when a tool like GitHub Copilot can perform this common operation in seconds? What AI can’t do is take a complex problem and design a system that will address any possible use case in the prompts. If you were able to write out all of these use cases, you’d essentially be writing the program yourself. And although AI might suggest a list of potential business strategies, no senior leader with business knowledge and context is going to ask AI for the best idea and run with it. I would take the view that AI will make work better for most people.
     
  5. Track impact via revenue. For me the most effective AI data metric to track is revenue. Are you actually making an impact or profit by building new products or driving efficiency in operations or scaling, whatever the business impact is supposed to be? That’s what you should be measuring. Sure, you can track the accuracy of a model, but from a data leader that’s more of a “Is your team doing a good job in executing the model?” question versus a “Is this a good thing to do?” question. You still have to measure the business impact. Ideally you can look at a revenue number and say, “Look at this. We did X, Y, and Z that generated this much revenue because of what we’ve done on the AI side, and this is how much it cost us, and let’s celebrate.”

AI: still an adolescent  
A lot of data leaders forget that the famous Alexnet paper only appeared in 2012. LLMs are a classic example of Jeff Bezos’s observation that “All overnight success takes about 10 years.” As far as we’ve come with AI to make it easier to create, share, and spread ideas, it’s still very early days. What it is an opportune time to do, however, is to get moving on creating impact with data AI in your organization. In fact, you might start by asking your favorite AI chatbot for a few suggestions. Where that’s likely to lead you is right back to ideating with your own highly talented colleagues.