Prophesying capabilities through business intelligence

01. 04. 20 John Chapman

Prophesying capabilities through business intelligence

The growth of big data and business analytics is an important development in business improvement. By being able to harness the information in disparate databases we have the potential to predict the future. At TouchstoneBI we have developed this future foretelling capability.

This is achieved through a combination of statistical modelling, including deterministic methods, parameter learning, Rubin’s Causal Model and quasi-experimental designs.

The first version brought together linear regression, probabilistic graphical methods and random forests. This operated by ‘constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees’

Now mathematical techniques (such as Box-Jenkins models) are based on the concept that future behaviour can be predicted precisely from the past behaviour of a set of data. We did recognise that these methods ignore the existence of disturbances; or external 'shocks' that may alter the data's future pattern, yet this represented an important first stage of development.

Next was the delivery of lower-level statistical techniques with JRules and Drools. The Drools brought a much needed ‘forward and backward chaining inference based rules engine’ which to be precise is ‘more correctly known as a production rule system, using an enhanced implementation of the Rete algorithm’.

To bring in the time element, stochastic process was chosen. This embedded mathematical knowledge and techniques including probability, calculus, measure theory, Fourier analysis and the application of the Bernoulli process.

Scientifically designed into the model was Markov processes. Traditionally in discrete or continuous time, that have the Markov property, which means the next value of the Markov process depends on the current value, but it is conditionally independent of the previous values of the stochastic process. In other words, the behaviour of the process in the future is stochastically independent of its behaviour in the past, given the current state of the process.

The magnus opus of the work, pushing back the boundaries of business intelligence capabilities, has been the blending of expert assumptions and inductive learning to identify causal relationships from observational data. This is where Rubin’s Causal Model was included. The framework is based on the idea of potential outcomes and the assignment mechanism: every unit has different potential outcomes depending on their "assignment" to a condition. Potential outcomes are expressed in the form of counterfactual conditional statements of the case conditional on a prior event occurring

The informed of you will know many statistical methods have been developed for causal inference, such as propensity score matching and nearest-neighbour matching which uses the Mahalanobis metric. These methods attempt to correct for the assignment mechanism by finding control units similar to treatment units; and it is with this final element of control we are able to prophesy future events. The model has been found to be most effective on the morning of 1st April between the hours of 08:00am and 12:00pm

TouchstoneBI is the only organisation with this unique capability and foresee a successful future for all organisations who engage with us, but hurry, because this limited time offer is only available until Midday on 1st April 2020!

  • Share on:
John Chapman

Written by:

John Chapman