Customer Story

CareQuest Optimizes their Data Workflows and Analytics with Databricks

What we did

Built a custom data infrastructure using Databricks to streamline data management and storage, plus enhance data science capabilities.

CareQuest data engineers are now proficient in best practices for data ingestion, transformation, and analysis, empowering them to independently handle complex data sets.

Industry

  • Healthcare

Technologies

  • Azure
  • ·
  • Databricks

The Problem

After separating from an affiliate, CareQuest needed to establish a robust and independent technological foundation to support its mission of improving oral health for underserved communities.

They aimed to build a comprehensive architecture stack, streamline data management and storage, and improve their reporting and dashboarding capabilities. With data coming from 20 different sources, organizing their data assets and enhancing their data science capabilities to handle unstructured data effectively became essential.

The Solution & Results

Databricks was selected as the data intelligence platform for CareQuest due to its comprehensive capabilities and its ability to handle complex data analysis use cases.  

With Databricks, we: 

  • Established a scalable cloud infrastructure capable of managing 13 terabytes of data, ensuring efficient data processing and security. 
  • Performed a seamless data migration to Databricks and Microsoft Azure, enabling the transfer of large data volumes into their new environment quickly and cost-effectively. 
  • Developed ingestion pipelines and transformation logic using SQL and Python to automate the continuous ingestion of Medicare and Medicaid research data into Databricks Delta Tables, creating a cohesive dimensional model for accurate and efficient reporting and analysis tailored to their specific research needs. 
  • Implemented Databricks’ Notebook orchestration and a CI/CD pipeline to facilitate collaborative data analysis, streamline data workflows, and automate data engineering tasks, reducing manual intervention. 
  • Accelerated advanced analytics capabilities by enabling analysts to work in R — a coding language familiar to their research backgrounds — ensuring everyone worked from the same governed data assets within Databricks rather than scattered data copies across various servers. 
  • Provided comprehensive tool training on Databricks’ capabilities, ensuring their data engineers were proficient in best practices for data ingestion, transformation, and analysis, empowering them to independently handle complex data sets. 

Talk with a data expert

Schedule a meeting with one of our data analytics experts who will review your data struggles and map out steps to achieve data-driven decision making.

Request a session