As we once again reach the holiday season our thoughts naturally turn to the year ahead. At this point each year I have always jotted down my thoughts and predictions for the BI industry in the year ahead, particularly for the mid-tier market. This year I thought I’d use my pulpit to share them with you.
The overriding theme appears to be “more” and “simpler”, as advanced technologies continue to trickle down from the tech behemoths into mainstream industry. I have broken down the trends into 4 areas; People, Technology, Infrastructure and last but certainly not least (especially for 2018) Compliance.
1. Continued Rise of the CIO – Over the last few years we have seen the rise of data and analytic specialists into more senior management roles as businesses continue to see the investment returns of a considered data and analytics strategy across the enterprise. This is most visible at C level where more Chief Information or Chief Data Officers are being appointed, with both analyst functions and the classic IT functions reporting into this position. The aim of this is clear, businesses want to profit from the vast amount of digital data being collected and want to see IT functions leveraging technology to meet this priority. The last year has had 74% growth in the number of CIOs on boards, but that still only equates to 31 in the Fortune 100 according to Harvard Business School, with the FTSE100 still lagging behind.
2. Rise of the Citizen Data Scientist – Beyond all other trends over the last few years, the ascendancy of data science has been most apparent. Large data sets being used to create decision trees and the rise of machine learning and neural networks has been all consuming in the data industry. However, so far these have required complex algorithms and highly skilled data and statistical specialists. This will change. As particular models and variants become more standardised and refined while the platforms become more able to self-refine, then tools will become more pre-built and open to the business analyst. For example, work on models for client retention and the probability scoring of each client, is now a standardised model available on IBM Watson. Indeed, Gartner now predicts that 40% of Machine Learning/AI tasks will be automated by 2020.
3. Socialisation – Social media has been the single largest change of the 21st Century. It’s rise was very much driven by the personal social network. However, businesses see the application of this social sharing as the basis for improved, more efficient working practices both within their business and their commercial operations. LinkedIn for example is now the 29th most popular website as of November 2017 (as ranked by alexa.com). Over the coming year expect to see more enterprise applications bring social functions into their platforms and indeed to layer a suite of applications within a single social wrapper, along with comments, tasks and cross application drilling all available. All the while more data is being collected about what and how staff are doing things, all feeding the data lake, all being analysed for bottlenecks and inefficiencies. Platforms such as Microsoft's Yammer and Infor OS will become the norm, not the exceptions.
4. Machine Learning – The Big Data trends of the last few years will now turn to be focused on making use of these vast data lakes. While many machine learning tools will start to become available to the user analyst, this wont see the end of the Data Scientist. IDC’s paper Data 2025 suggests that we currently create 12.3 zetabytes of data per year. By 2025 they predict this will have increased tenfold to 163 Zb. What will we do with all this data? What we won’t be doing is ignoring it. IDC forecast that spend on AI and Machine Learning from Enterprises will reach $46bn by 2020. The trend is increasing with more work being done than ever to truly help businesses understand issues such as price sensitivities and logistical challenges.
5. Automated Data Prep & Insights – As the volume of data captured in Enterprises vast data lakes increases the more important it has become to be able to apply good data practices to them at the semantic layer. However, with the ever increasing demands coming from both data volumes and from ever more sophisticated use cases, the heavy lifting of what have become relatively well understood, process driven tasks has been handed off to standard procedures. However, full automation has always been a little out of reach. The last few years have seen this change. With Birst having recently been awarded patents for it’s Automated Data Refinement algorithms, it currently leads the field in the automated data refinement initiatives, but with Oracle and Microsoft in particular not far behind. The aim being to be able to point our analysis and visualisation tools at large data sets and have the tool build the most appropriate model to make our data analytically ready. This will also lead to the platforms to provide us with more useful insights into trends, outliers and potentially missing data points (and auto compensate for them if we so wish).
6. Natural Language Processing – Already becoming common in BI tools such as MS PowerBI, we expect the use of natural language processing to become more widely available in the Enterprise Software space. The large neural networks and deep learning models required to undertake this are becoming more common place and are no longer the preserve of the tech behemoths such as Apple’s Siri and Amazon Alexa. IDC expects that 75% of users will use an Artificial Intelligence based personal assistant within their Enterprise Applications to give quicker access to functions and data to complete their tasks as well as classic PA functions such as task scheduling and diary/process control.
7. Guided Visualisations – The AI that delivers insights and interprets our natural language requirements will also allow for the automated prompting of the best visualisations for our selections of measures and attributes. No more pie charts with 200 slices! Consumerisation of data and the ubiquitous use of AI will lead to the Amazon and Googleisation of the interface - look for nudges, prompts and search anywhere within your Data Visualisation tools.
8. More Clouds and Lots of Them – Cloud is everywhere. Your photos on your phone, your desktop office software, our personal music collection. Why should business be any different? The next year will see more businesses moving to the cloud and SAAS software models. However, they’ll have lots of clouds. With lots of different API’s. Gartner predict that 70% of businesses will be multi-cloud users by the end of 2019. So the good governance of data will become more important, as the need to stage, cleanse and transform data to make it analytically ready will need to incorporate flows from multiple cloud environments, probably into a cloud environment of its own. The world of a fragmented cloud isn’t going anywhere.
9. Lots of Things and their Locations – More and more sensors and devices will continue to be connected to the internet, all pulsing data and all geo-located. This trend will accelerate in the business environment, picking up more physical plant, more structures, more production lines and more processes controlled and analysed. Asset Management tools will become highly entrenched within the data management field. Geo-analytics may become the next big trend in business analytics with people looking for surprising correlations or outliers based on locations.
10. GDPR – the governance obsession of the year – is not going anywhere. Businesses continue to be under-prepared and lacking an appreciation of the work required of them. There will be more data leaks, more governance and all will be covered under the auspices of GDPR. More time will be invested into Master Data Management and Data Lineage. There will also be a knock on effect into Data Insurance. This is now becoming a major line in the specialist markets such as Lloyds of London as people realise the risks to their businesses from poor data control or the loss of data systems.
In summary, the star of the show in 2018 will be AI and Machine Learning but from a business perspective this will mostly appear in the form of the trickle down effect of proven methods.
For more Business Intelligence insights from Paul and the rest of our expert team, keep an eye on our blog, or contact us to find out how we can help you gain better insights into your data.