How to Achieve up to 98% Accuracy in Forecasting Models: Stylus Case Study

Solution

• Dedicated Python scripts
• Machine learning
• Power BI for visualization
• Various forecasting models

Why Machine Learning?

Using forecasting models based on machine learning instead of the intuitive calculation methods allows increasing accuracy in:

‍

• Advertising budget planning
• Calculation of the call center workload
• Warehouse workload planning and logistics

Step 1. A Test Forecast

To forecast each metric in the most accurate way, we took a data set from the CRM system and Google Analytics account for the past 2 years. After studying the data, we made test forecasts to predict the number of sessions and transactions for the next three months and two years. This allowed tracking all the abnormal periods that increase the error in forecasting.

‍

We additionally analyzed the key components, such as trend, weekly & annual seasonality.

‍

Trend analysis showed that a significant increase in sales for the retailer started in November 2018. Weekly and annual seasonality charts allowed us to identify the periods of growth and fall in the number of transactions and predict the frequency of orders by month with an error of 10-15%. To solve the retailer’s tasks, we decided to achieve more accurate indicators, since the everyday data was needed.

‍

Step 2. Achieving More Accurate Transaction Indicators

‍

To achieve more accuracy, we refined forecasting models and created a dictionary with all the holidays, as well as periods of increased demand (Black Friday, Christmas, etc). 

‍

This helped to reveal one more pattern: promo campaigns launched close to the period of high demand impact the dramatic trend changes. At the same time, this fact had almost no impact on the weekly and annual seasonality.

‍

We also excluded from the report the abnormal periods (server being down, Google Analytics counter interruptions and so on) that affect the accuracy of the results.

‍

After all the adjustments were made, we managed to forecast the number of transactions with an error of ~8% per day, and ~2-5% per month *The error of the forecast is growing with the increase of the period forecasted. More than 3 months – the accuracy drops exponentially, up to 80-50%.

‍

‍

‍

‍

‍

Step 3. Logistics Forecasting

‍

‍

‍

To distribute the warehouse workload, calculate the number of daily shipments and forecast the delivery of orders, we worked out a data set from the CRM and predicted the total number of orders, dividing them by country parts. We also created a dictionary of locations by region. 

* For some locations, the data was insufficient, so we combined them into a separate sample “The other parts of the country”

Step 4. Visual Representation of the Data Acquired

To make the perception easy, we uploaded all data on the predicted transactions, calls and orders to Power Bi and presented it to the Stylus team in the form of interactive dashboards. 

A forecast of transactions and requests to the call center

‍

Logistics Forecasting

‍

‍

Results 

90-98% The accuracy of the developed forecasting models

~5% The average error of short-term forecasting models (per quarter)

Such high accuracy indicators were achieved thanks to: 

‍

• The presence of sufficient volume of educational selection (historical data on the metrics of interest for 2 years) 
• The presence of up to 50 values of a certain metric per time unit (hour, day, month) 
• A relatively high level of variance during a certain time period

‍

What’s next?

• Apply models to other metrics (number of users, number of payments)
• Automate prediction models to automatically retrieve and update data