We will analyse the existing workflows at the Data Science and Data Processing teams, where we will develop automations. In both areas the workflows are long, sometimes tedious, and implies too much manual execution and supervision. First, data needs to be cleaned up to make sure the model gets good figures so it can show results with 90% accuracy. Nowadays, 90% of this workflow is done manually. Second, data science team needs to find the right model after training thousands of them, taking in consideration price elasticity, price cross-elasticity, cannibalization, competitors pricing strategy, special promotions, seasonality and pass through prices to final customer, among others. Currently 60% of the Data Science team workflow is done automatically.
In this project we have identified potential automation improvements and define a development and integration roadmap to reach Athena 2.0 beta version, including the technical validation methodology. We explained the process for the automation of execution of the current scripts used to generate the pricing and trade promotions optimizations. This made automatic 80% of the current manual tasks for Data Processing team and 85%-90% for Data Science. Also, we have defined and develop automate deep quality assurance modules (Semantic QA), that has allowed to early detect any problem based on previous iterations.
In addition to automations, we have built a recommendation engine to find the most relevant, actionable insights for the customer and predict their interests using Reinforcement Learning (AI) techniques.