Keys to a Successful Data Warehouse Project

Learn More About Data Warehousing

Most organizations know that a solid data warehouse is an essential part of most business intelligence systems and the foundation to build meaningful analytics. However, data warehouse projects often seem daunting and don’t meet expectations or fail to achieve adoption. In this blog, we give advice on how to ensure your data warehouse project is a success.

Why Data Warehouse Projects Go Awry

First, let’s break down why data warehouse projects have a bad reputation:

  • Poor Requirements: Many times requirements are meticulously documented and cataloged, but they do not address the business objectives; instead they are created to demonstrate progress and complexity of the project.
  • Big Bang Approach: Multi-year data warehouse projects are risky, expensive, and no fun. Often, data warehouse development isn’t segmented into manageable, relatively short iterations.  This failure to quickly iterate and frequently deliver business value often leads to loss of project momentum and executive sponsorship.
  • Talent Deficit: We see that teams new to data warehousing make mistakes under pressure to deliver. While this is human nature and part of the learning experience, a data warehouse project requires experienced project members.
  • Data Quality Issues: “Garbage In, Garbage Out”. Successful data warehouse projects must include time up front to assess, prioritize, and remediate necessary input data quality issues.  Failure to address significant data quality issues can lead to loss of trust in the data for end user groups consuming outputs from the warehouse for the first time.

Steps to Achieve Success with Your Data Warehouse Project

Here are some things to consider for a successful data warehouse project:

1.) Start with skills. Assess the skills of your team.  Are they skilled in data integration and modeling?  Are they trained on new technologies and approaches?  Have they worked on similar projects, both in domain and scale? Do you have team leads who are capable of mentoring and guiding less skilled staff?

Address skill gaps by getting the right people from across the company.  When not available internally, reach outside and find skilled people to help.  Partner with a consultancy whose core competency is data warehousing.

Do: Assess skills related to Requirements, Architecture and Design, Delivery, Testing, and Project Management related to Data Warehousing.  Partner with consultancies when necessary to fill skills gaps and provide a co-development model in which your internal team is “taught to fish”.

Don’t: Omit critical project roles or stretch current staff outside of their areas of expertise due to lack of resources.

2.) Identify business requirements with corporate and departmental objectives in mind. At this stage, it’s not about the data; it’s about identifying business needs to operate more efficiently and make data-driven decisions.

Do: Address your reporting and analytic gaps as a priority. What new capabilities can we enable?  What are the “expensive” business questions that can’t be asked of your current data and systems?  What, in a perfect world, should be measured (regardless of what is currently available)?

Don’t: Just port all your existing reporting requirements to the new platform.

3. Assess data requirements. With analytics requirements in hand, identify the sources of data needed to achieve each requirement.  Asses the quality of the data sources available and identify any data remediation that may be required for each source.  Compile a Data Warehouse Bus Matrix and conceptual data model – both will become core elements of your data warehouse requirements.

Do: Leverage Data Discovery to validate and assess data assumptions.

Don’t: Waste time on data for fringe use cases or low priority analytics (which is easy to do!).

4.) Assess the Bus Matrix and create a roadmap. Use the Bus Matrix to help prioritize data sources.  Take your highest priority analytic requirements and identify all required sources.  Create an incremental roadmap that delivers the highest value analytics first. Each increment in the roadmap should be manageable in scope.  If the scope is too big right off the starting line, reprioritize so that you can implement low effort-high value items first.

Do: Leverage the Bus Matrix as a tool to communicate and gain consensus on completeness and prioritization.  Find a quick win or two to begin with, set the stage for further expansion, and gain momentum from there.

Don’t: Be too aggressive with scope.  Creating momentum and success early creates opportunity in later phases.

5.) Address the architecture. Identify a technology stack that will meet your long-term business needs.  A successful data warehouse should have a lifespan of potentially many years.  Plan to build out the skillset necessary to run and operate the data warehouse, or select a technology stack you’re familiar with.

Do: Get an outside opinion.  Data warehouse experts will expedite project completion and accuracy.  Conduct a “bake off” to compare various tools (database platform, integration, and business intelligence / reporting) using a subset of your own data.

Don’t: Select a tech stack because it’s the newest coolest technology.

6.) Manage to completion. Each phase of the roadmap should be delivered to completion as if it were the last step in the roadmap.  Failing to do so will affect later phases and sets a precedent that “done” doesn’t mean “complete”.

Do: Define clear success criteria for each phase and inspect to completion to ensure that you are not reporting false velocity.

Don’t: Dramatically change scope during a sprint or phase. Some change is normal and expected, but too much change will jeopardize the phase and create risk of running over budget and behind schedule.

7.) Measure Success and Communicate it. Each phase of the Data Warehouse project should be creating value.  Define, measure, and communicate the value.  A project that is delivering incremental value will create momentum and increase executive sponsorship.

Do: Measure value in dollars, time saved, insights gained and the value of those new insights.

Don’t: Focus on tasks completed; focus on the business value instead.


Our Data Warehousing Services

Whether your data resides in spreadsheets, various operational systems, “data landfills”, or you already have an enterprise data warehouse, Analytics8 can help you transform it into consistent and useful information so you can make better, more informed decisions.  Read about our data warehousing services here.

Read about our data warehousing work with Guggenheim.

Analytics and big data. It's what we do.

 

Contact Us

National Office Telephone | Mon-Fri 8:30am-5:30pm CT