Modern Data Platform Evaluation
The first step was to assess the client’s current data landscape and state of applications and their demands for data integration based on business needs.
We looked at their business requirements for data and functionality, from the key select sources including WolfPack, Guardian, and select unstructured manual data sources, and organized the data into subject areas to serve as a semantic layer for all data available.
With an understanding of their current state, we then began evaluating viable solution options for a target modern data platform. Based on feedback from the client, we presented two options with a focus on cloud implementations that support BI and analytics — an Azure-based data architecture and a Snowflake Data Cloud solution.
Then we provided the client with a complete assessment deliverable that included our findings of their current state and viable options for the modern data platform, including:
- An outline of solution options, components, benefits, and impacts
- An outline of the data architecture
- Estimated costs for development and deployment
- Anticipated operational costs
- Side-by-side comparisons of key features and considerations of the two options
We also provided the client with a high-level roadmap of prioritized next steps, which included the creation of POCs to test the business use cases against.
Modern Data Platform Proof of Concepts (POCs)
Armed with the information from our initial assessment, the client asked us to execute a combination of POCs to demonstrate how these modern Azure-based cloud platforms and technologies would support the company’s BI and analytics needs.
To create the POCs, we defined the modern data platform reference architecture within Azure with options for specific technologies, including native Azure components (e.g., Azure Data Factory, Azure Data Lake, Azure SQL Data Warehouse, Azure Analysis Services, PowerBI, and Azure DataBricks), and Snowflake.
Building the POCs included:
- Configuration of Azure subscription and services
- Design of structures in Azure SQL Data Warehouse
- Creation of tabular model in Azure Analysis Services
- Execution of data pipelines into Azure and Snowflake
- Creation of a Snowflake database & designated views
- Build out of data pipelines between Azure services and Snowflake
- Creation of analytics model using Spark with Azure DataBricks
Once the platform was created, we demonstrated BI reporting using PowerBI against the POC platforms and the execution of machine learning use cases using Azure ML studio and custom model development.
We then tested multiple use cases for data ingestion, BI, and analytics. We validated various use cases for viability, including:
- Financial reporting using Dynamics AX data that is integrated with budgeting and pricing data
- Supply chain optimization
- Predictive maintenance