Today’s data environment is one defined by compressed transformation and innovation cycles. This intensity only picked up during the recent pandemic, when we all had to pivot and quickly change the way we were operating. Adoption times for new technologies are shrinking, and when you add GenAI to the picture the pressure to move faster only increases further.

Yet robust data integration and interoperability – when enterprise applications can interact with each other and exchange data – lies at the center of any initiative to become more innovative. The faster you can pull together all your sales data, service data, and various customer data sources, the easier you are able to provide superior experiences to these audiences. Recent global research we’ve conducted shows that companies with high levels of data interoperability grew revenue six times faster than their peers. 

So how is data getting in the way of innovation rather than accelerating it? Here are six dynamics I see a lot: 

  • A complex integration landscape. A variety of different tools and multiple point-to-point integrations make change and evolution in data integration difficult
     
  • Legacy integration platforms. These platforms typically don’t allow easy access to different systems or data sources, which means you can’t move faster. Organizations may be running on older platforms that haven’t been upgraded in years – building technical debt and making it harder and more expensive to do so now
     
  • Data silos. In many large enterprises there may be up to 1,000 different applications. Teams like Marketing, Sales, and Products all have their own systems and information, but typically don’t share or aren’t even aware of the data in other parts of the business. And any situation where data isn’t used to its fullest extent lessens its value
     
  • Security and governance concerns. In many organizations, opening up access to data sources can understandably trigger security and governance concerns. While the concerns absolutely need to be addressed, this activity can be seen as slowing down the pace of innovation  
     
  • Skill gaps. Locating GenAI skills is all the rage these days, but when it comes to integration, bandwidth to integrate different data sources is often very limited as well. Even when these teams exist in enterprises, they are often managing a high volume of requests and demand, and end up becoming a major bottleneck
     
  • Manual processes. Every organization has manual processes that are slow, inefficient, and not scalable 

What is modern data integration? 

What makes a data integration platform modern depends on what you’re looking for. Generally, though, most integration platforms you consider should:

  • Be able to unlock, access, and interact with diverse set of data in a more real-time manner as compared to traditional batch-oriented processes
  • Benefit from technologies like low-code, no-code, and self-service, so using the system is not limited to just skilled IT resources. Key to this broader user profile is moving away from point-to-point integrations between applications that use custom code
  • Take advantage of automation capabilities that can connect you to more sources and allow you to scale better while helping to streamline data mapping and error handling
  • Speed up access to data through approaches like zero ETL or zero copy. A lot of organizations are moving away from the large batch jobs that used to define their approaches, and the data sources are now capable of sharing data over standard protocols
  • Use APIs as reusable, interoperable components to connect applications and systems without the need to build new code each time. These digital building blocks can be configured and reconfigured rapidly to “compose” new solutions to evolving business needs

For many organizations, it’s an inevitability that they will need new integration tools or a new data integration platform. This can’t be done with the flip of a switch in large enterprises. 

A better approach is often picking a subset of projects; for example, you inform your team that all new integrations will be done on the new platform. By prioritizing use cases, starting small and moving through those first few projects, you begin to build valuable experience. Data integration becomes easier, and you create more reusable components so that the next projects that come along are able to take advantage of your previous efforts. Reusability is a key driver behind integration as well, so you want to take advantage of that, too. Leaders in organizations are now tracking availability of core data concepts over APIs as a KPI.

Back of the line

If you’ve had any experience in a data integration effort you may be asking, “Why is data integration often the last bullet item on a digital transformation checklist?” In my experience, people simply don’t realize how hard it is. With a more limited team typically doing the integration work, the weight and scale of the lift are often underestimated. The reasoning I hear is often, “It’s just connecting a few data sources: how hard could it be?”

A complicating factor is that as other parts of a project plan may be delayed, the go-live date often stays the same. This can cause a lot of unnecessary stress for data integration at the end of a program. Before you begin it’s better to understand:

  • The dependencies for all source and target systems 
  • What needs to be built and what is available on those systems before integrations can be designed and executed
  • What level of effort is involved to execute design and build robust integrations
  • Who is needed to realize this effort, so it can be planned for upfront
  • What could be gained if you integrated with other data sources, especially those in the cloud

Remember that every source will be in a different format, available via different methods, and that enterprise integration tools need to be able to support this reality, whether you’re building APIs to connect, messages to communicate, doing any sort of ETL, sending out partner communication, or performing last-mile automation where you might be screen scraping off of something just to have access to it.

5 data integration questions to ask about your project 

Every organization in this compressed transformation era is being forced to update and become more modern. It’s all about how you can connect faster and in a more scalable way to give yourself as much access as possible.

So what’s your data integration plan? What are your data integration requirements? Here are five questions you should be asking right now.

  1. What is our current data strategy? What’s the role of data integration in it, and when we did last refresh it? 
  2. What’s the current interoperability status of our systems so they exchange information in real time, and what do we need to do differently? 
  3. How are we maximizing access to all data across the enterprise, given these data live in many different sources and don’t all work in the same way?
  4. How are we working with our enterprise architecture team and central IT organization to draw up a strong action plan for data integration? 
  5. How do our current data tools map to the capabilities we need for the next generation of applications and business needs? Is anything missing? Are there new tools we should know about?  

Of course, one of the most common data integration questions we hear from any client these days is about what they should be doing with AI. Yet the same data discipline and questions apply, and should be applied, to pull people back from a pure outcomes focus to think more deeply about what the data strategy behind your AI goals should be.

Although a data foundation isn’t seen as the most glamorous part of IT or product groups, it’s essential for data leaders to remember that it’s the common base on which everything is built. It simplifies the work you do with greater cost efficiency. It even makes data governance easier to implement. 

If you think of your enterprise as a physical thing, even if so much of it these days is virtual, your data is the foundation on which this enterprise rests. The more integrated and interoperable that data is, the greater the heights you can take your organization