Achieving corporate agility through data & analytics

Most organizations are brimming with data. Information about customers, products, services, etc. All this data can be a treasure trove for companies by allowing them to customize customer experiences and improve internal efficiencies. But, without the right strategy and technology, data can become nothing more than a digital paperweight, providing little valuable insight for companies to act on.

Our client, a large real estate investment firm that manages shopping malls across the nation, wanted more from their data. They were burdened by extensive manual data entry, multiple systems that held different information, and delays with reporting and analytics. Our client was also dealing with data storage. With their on-premises infrastructure about to reach capacity, they knew it was time to act. Like many companies, they found themselves asking if now was the time to move to the cloud.  

Additionally, in a number of their shopping malls, they provide free WIFI. This is achieved through having multiple wireless access points installed throughout the mall. The installation and maintenance of these access points is managed by a third-party provider.  

They also had a third-party provider that used smart phones to “ping” data from the mall’s access points to estimate the number of people in the mall (foot traffic) on an hourly and daily basis. The amount of data received and processed from these IoT devices exceeds one billion records per day, and the third-party solution required a significant Google Cloud implementation.  

The cost of this implementation was paid for by the client along with the fee for the service, costing upwards of $25,000 per month. In addition, because the algorithm used to derive the foot traffic counts was proprietary, there was no transparency on how the calculations were done. This led to a general mistrust of the data that was being received.  

Adding to that is the fact that smart phone manufacturers are periodically updating how they interact with wireless networks, requiring periodic updates to the algorithm. When this occurred, there was often a large delay — weeks and sometimes months — before the third party addressed the changes. 

They needed to figure out a way to gain valuable, actionable insights, without the burden of the cost they were paying and needed the right tools and expertise to do it.  

Cloud Solution

We started with an assessment of their existing system and data sources to determine requirements for a data platform that would better provide stakeholders with the insights they need. Our client was looking at different cloud options and wanted to ensure their platform decision would work with their existing systems and allow them to improve their current processes. More than just a new implementation, our client needed the right data strategy in place that would allow them to meet their new business goals.

Based on interviews with stakeholders and the results of the platform evaluation, we ultimately recommended a hybrid cloud model that would work with their existing infrastructure, utilizing Azure as a cloud solution and the Snowflake data platform. We created a roadmap that prioritized options for building out their data and technology strategy and a plan for getting from the current state to the updated future state.

Implementation & Proof of Concept

Armed with the recommendation and roadmap, this investment firm was ready for implementation. We worked with them to:  

• Build out the new data architecture, data model, and ingest processes 
• Identify and integrate new data sources into their analytics environment  
• Create and refine data governance processes and information  

They made the decision to replace the third-party system with an in-house solution. This decision took the raw data from the access points and processed it based on an algorithm developed in-house and gave transparency into the calculations that would allow them the ability to directly update and enhance the process when required.  

This solution would also better integrate the data into the Snowflake data warehouse environment allowing it to be used with other data sets such as marketing and sales to assess the impact of promotions. The decision was also based on an assumption that an in-house solution would have at least somewhat lower operating costs, or at a minimum, the costs could be better managed. 

We worked with the client to perform complex data analysis on the incoming raw data to develop and test a new algorithm for estimating mobile device counts and foot traffic numbers.  

This included working with the business users to define requirements and parameters for what should and should not be counted (for example: don’t count visitors that are there less than 15 minutes) — input that they did not have with the third-party solution. Once the algorithm had been defined, Fusion built the solution to implement it in Snowflake, including tuning the process for optimal speed and cost savings.