If your current data infrastructure doesn’t “spark joy,” read this explainer before investing in an Enterprise Data Warehouse. We argue that firms should think about business objectives, analytics, required data flows, and tools – in that order – and that modern tools and services offer potentially big benefits over a traditional data warehouse given management’s goals for the business.
Enterprise Data Warehouses (EDWs) represent long-term commitments by businesses to a specific set of metrics and supporting data structures. Will those metrics and data structures be optimal? This depends on the business’s objectives. Building an EDW before understanding exactly what the business needs risks wasting significant resources. And ultimately, a traditional EDW may not be necessary given recent advances in other scalable and cost-effective data integration and storage services.
Middle-market firms at a crossroads
The question of how to optimize data infrastructure is particularly pressing for middle-market firms whose recent history of rapid growth and forthcoming business model evolution puts them at a crossroads. The data flows that such firms have in place tend to be the ones that were needed to allow the business to function operationally as it grew to its current size. But they are no longer adequate.
As management considers what needs to be done for the business to shift gears, they know data is important and that other businesses seem to have more “mature” data operations. Teams at this stage often contemplate making large investments in data infrastructure and hope this will help accelerate growth. Unfortunately, this alone is insufficient.
What’s missing is not data infrastructure per se, but the ability to continuously evaluate the business across numerous dimensions and drive innovation. (See our related post on how data strategy sets the stage for innovation). Gaining this ability often does require some investment in new data flows, but those flows should be tailored to the new analytics that they will ultimately support, just as the analytics should be tightly aligned to business objectives.
Start with an alternative approach
Optimal data flows and infrastructure follow from asking key questions and identifying the methods that will be used to power them. Questions like, “How should the business be scaled?” “What aspects of the current business model could be stressed by scaling in this way and what can be done to eliminate that risk?”
Value creation plans may suggest specific directions to test, but are often intentionally vague. Blanks need to be filled in and adjustments will almost surely need to be made along the way. Agility will prove very valuable in whatever data management solutions the business utilizes.
First, make a strategy centered on specific business objectives. Consider the kinds of analyses and tools needed to address key objectives and evaluate progress. As part of this, consider how the management team and board might want to interact with them.
Next, determine the minimal set of data needed to make these initial analytics work. Beware of the confusion and limitations that existing data systems and aggregations can cause: we frequently encounter teams with conflicting understandings of their own firm’s data definitions and limited visibility into raw data sources. Even if such confusion and limitations were unimportant historically, they can become very problematic as the business gets more reliant on metrics composed of ingredients from multiple functional areas. Consider sourcing new analytics from the rawest, most granular data available rather than from existing data flows and aggregations. Go back to each siloed transactional database, identify the most foundational relationships between them, and then use those relationships to design data aggregations with specific analytics in mind.
Finally, get data flowing. Modern, cost-effective apps and services can be deployed to quickly orchestrate the data flows for initial versions of new analytics and tools. The same apps and services are fully scalable as teams begin to utilize their new tools and start providing feedback that leads to useful refinements and enhancements.
Still considering that Enterprise Data Warehouse?
Businesses still considering an EDW should perform an ROI analysis that incorporates realistic assumptions about costs, timeline and goals vs. needs. A large part of the cost of a traditional EDW is Opex (additional personnel and/or vendors). Benefits are unlikely to begin to accrue within the first year of the project, and this timeline must be balanced against business objectives and the need to maintain agility and competitive edge. The resulting ROI for an EDW should be benchmarked against an alternative approach that leverages contemporary, cost-effective, and scalable data integration, storage, and analytics services.