The term ‘predictive analytics’ is often used as a generic or umbrella term for advanced analytical models. Predictive analytics is used in many business and industrial applications, and business functions. It uses techniques, either alone or in combination, from data mining, statistical modelling, machine learning and operations research, including optimisation, to help organisations improve their performance and give them competitive advantage by providing insight and understanding into their performance. The insight gained from the models can then be used to help formulate and implement new policies.
There are many types of predictive analytics model, two of which are forecasting models and simulation models. Simulation models are used for comparing the effects of a range of alternative polices to determine the optimum policy. The results of the models must be presented in clear and meaningful ways, and accessible by non-technical users at all levels in the organisation for them to interpret the results from a business perspective and then work out and implement the necessary operational and strategic changes.
The ever-increasing amount of data, and the readily availability of powerful hardware and advanced software make predictive analytics accessible to a much larger number and wider range of organisation than was the case only a few years ago. Furthermore, more demanding and competitive economic conditions have only added to the imperative for organisations to extract as much commercial advantage as possible from all the data they hold - an asset that until a few years ago was not used to its full extent, if at all.
In their book Competing on Analytics: The New Science of Winning (2007), Thomas Davenport and Jeanne Harris describe four requirements for organisations to use analytics successfully:
- senior management must be committed to analytics and provide the leadership to embed it in the organisation
- the analytics must be used in all parts of the organisation
- the organisation must have the appropriate hardware and infrastructure to support the analytics
- the organisation must have people with the analytical, modelling and business skills required to get maximum business and commercial advantage from the analytics.
Many organisations believe that business intelligence and predictive analytics are the same whereas in reality they are quite different (predictive analytics is described above).
Business intelligence, often called reporting, is used to present historical data or the results of modelling the data in a range of tables, graphs and charts. In contrast, as discussed above, predictive analytics uses historical data to develop models for making predictions. Thus, a clear distinction between business intelligence and predictive analytics is that business intelligence looks backwards to present what has happened whereas predictive analytics looks forward to present what might happen under a range of different scenarios.
Black Box Software
Black box software is software for which little or no information about the algorithms in the software is provided. This lack of knowledge severely limits the ability of users to gain maximum benefit from the software and can lead to inappropriate models being used and invalid conclusions being drawn. In the opinion of PAM Analytics, working in such a blind way is bad practice and treats modelling as a prescriptive process rather than as a creative task. To quote Alexander Pope (1688-1744) in An Essay on Criticism (1709) ‘A little knowledge is a dangerous thing'. To paraphrase ‘more is better’.
Advocates of black box software argue that it can give better results than non-black box software. This may be true for the training datasets but it is very dangerous to apply models that are not fully understood or whose limitations and assumptions are not known to new data, and then to use the results of these models as the basis of an organisation’s objectives and strategy.
The problems associated with black box software can be overcome by applying a robust approach, such as CRISP-DM (see The CRISP-DM Methodology inModelling), to all aspects of modelling projects. It emphasises the importance of understanding the data, the business context of the problem and how they relate to one another, and then selecting the appropriate models.