Amidst all the buzz in the space of business intelligence with acquisitions, consolidations, and the likes, analytics is definitely taking a new direction. Will this be the end of business intelligence (BI) as we know it or will this ongoing consolidation of BI providers only make the entire space of data consumption stronger with increasing opportunities? Is AI taking over the space of insights? Does digital process re-imagination have any relevance to how the future is shaping up for business intelligence are some questions that we will ponder over today.

• The transformation so far –

Source system > OLTP > ETL > DWH > Data Mart > Semantic Layer > Reporting
Source systems > Data Lakes > Catalogue and Virtualize > Semantic Layer > Reporting
Source systems > Cloud Data Lakes > Snowflakes > Semantic Layer >Self Service BI
Source systems > Intelligent Data Platforms > Web-based Search Driven Insights (ML Based)

While the transformation of the data space took at least three generations to get to this point, put loosely of course, the consumption space is going in for more of a big bang change. The reason being most of the businesses have already exited the data centers or are doing so in a frenzy to move to the cloud. IT teams have consciously made sure that the way data is available for business should take care of their consumption needs and are keeping it as open as possible. Now with the advent of next-generation search-driven insights tools which are poised well to implement ML to give end-users the insights they want, the whole point of business intelligence will cease to exist. The only catch here is digital process re-imagination; as long as business processes are changing, ML can’t reach a stable situation and thus BI might still hang in there for a while.

• The two schools of thought – Let us segregate the happenings in the market today. One school of thought is that of creating intelligent enterprises by using embedded BI. The front end business applications will in greater probability integrate the capabilities of the insights platforms to the front-end systems trying to make the end-to-end process intelligent. The other school of thought is that of a data/insights platform. Since the customer is already moving to a cloud-based setup, it only makes sense for them to use an intelligent data platform that is equipped to give the needed insights in one suite, while the channels may be different for each.
• The next big disruption – This is one area which will have a bearing to the way data consumption on the cloud is going to look in the future. We all will agree the way we use the cloud infrastructure is going to be a determinant of the success we create. This is where the angle of micro-services comes in. Imagine a Black Friday situation where the load on data and intelligence infrastructure more than quadruples. So what can be done to plan the infra. While it might sound easy to just scale up BI servers for those few days, it actually is a tricky game getting things to sync up so soon. Now imagine a situation where an entire prediction model is sitting as a server-less micro-service built on Python and some near skills and these just give a final recommendation to the BI Server. This will enable optimal planning for the BI infrastructure with immense possibilities.
• Now as BI Engineers – The trick is to just change the perspective. All this while as BI engineers we have continued to think of problems as dimensions and facts, and there is no denying this. The expectations of the businesses on the other has been shifting to tying insights to business outcomes or to impacting business processes. So all we have to do is the think about our work as business challenges where a set of variables are affecting an outcome and do whatever possible to solve this, be it data science, data wrangling or just about anything. Also, there is a major shift in the way people want to look at insights, the attention span of customers is coming down significantly. They don’t want reports/dashboards and the likes but maybe just a narrative or an answer. That’s where the chatbots and the Insights Rockets (NLG Tools that generate narrative on dashboards) of the world are becoming prominent. So, this means no more drag and drop, the sooner we refresh our coding and SQL skills the better it will be for us.

To conclude

So in essence, BI, as we know it, is definitely on the chopping block. But it will stay relevant as long as digital revolution is changing business processes and as long as we are inventing new modes of serving data to consumers. It was MIS, it is BI and it will be insights, but the essence, will live on.