Implementation of Predictive Analytics

So what is needed to implement predictive data analysis? The predictive analysis process can be described as a multi-step process in which raw data are verified and processed to generate predictions based on a mathematical model.

The first step is the task statement. Before creating a predictive model, we should decide what output indicators we want to get, for what period, in what dynamics and by what categories. Moreover, for subsequent data collection, we need to formulate hypotheses about the expected influence of certain factors on the analyzed indicators.

After the task is set and the hypothesis is formulated, we can proceed to collect historical (retrospective) data both for the analyzed indicators and predictors (factors). The computing power of companies has been growing in recent years, they collect data on their internal indicators and processes, and on external factors, therefore, it is important to determine the required data sources among the available corporate ones and, if necessary, supplement them with external ones when collecting data.

When collecting data, we should take into account their volume (depth of retrospective) and quality. Efficient analysis and objective conclusions are possible subject to a large amount of relevant data. As for the data quality, the data must be accurate and error-free, that is, such that they can be relied on and correctly interpreted. The collected data are verified and “cleaned” from errors, such as duplicates, outliers, adjusting incorrect formats, and so on. 

Once all the data are collected, we can start handling them. It is the research (exploratory) data analysis. On this stage, we identify patterns and hidden relationships in data, the so-called "insights", and build a predictive model is built. When building a model, it is important to identify predicting factors that affect predicted indicators / phenomena. For example, to build forecasts of demand for products, we can consider such predictors as weather conditions, seasonal sales, days of the week and holidays, seasons and time of day, as well as the price level, monetary incomes of the population, etc.

It should be noted that for models containing socio-economic indicators (variable parameters of the external environment) among predictors, it is advisable to build scenario forecasts (depending on alternative scenarios of situation). Scenario forecasting allows you to use several options for the most likely scenarios (including baseline, pessimistic and optimistic), that will enable a company to quickly refocus in the future in changing conditions. By way of example, our company has developed a predictive model designed to predict regional labor force balances and unemployment rates by industry.

We determined the scenario and control parameters, developed the analysis scenarios and built a forecasting model based on the methods of correlation and regression analysis. Among the scenario parameters, we used such indicators as changes in the population size by age groups, the index of physical volume of production by industry, investment, and the index of real disposable money income.

At this stage, statistical methods and data mining methods are applied. The current trend is the use of machine learning. Regardless of the method used, an important step in building a predictive model is its testing for assessing the accuracy of its outcomes, and correcting the model, in necessary, and further re-testing it.

Based on our experience, we would recommend developing predictive models within a single analytical system, namely, together with design of a single data warehouse, and, if necessary, setting up automatic data download from the websites of statistical agencies, introducing tools for initial data validation, developing input forms for scenario parameters and reporting forms/dashboards to display the values of forecast indicators.

Our company will be happy to help you to develop such systems.