Project overview
Shortly after deciding to use a cloud-based AI solution, our client faced a challenge of interacting with the AI platform by exporting event data from a Big Data platform and Data Warehouse to the cloud and next, ingesting produced predictions from models back to the Data Lake for further analysis. Our client needed a generic solution to prepare and produce thousands of events daily/weekly/monthly, regardless of transformation logic, and support various techniques of event calculation.
Client background
A major operator and service provider in Poland – part of global international mobile communications group.
Goals
Our client needed a solution that could support thousands of data pipelines and calculate events using a huge volume of data. Security was paramount, so the process of exporting data to the cloud had to include pseudonymization. Additionally, we needed to implement configurable data quality checks as part of the process.
