More and more organizations see the benefits of Business Intelligence and Analytics. Thanks to continuous innovation, more and more data is becoming available. This leads to useful and surprising insights into both the private and public sectors, which use the acquired information in increasingly smarter ways. In practice, Passionned Group forecasts the following seven Business Intelligence trends for 2016.
1. Data Ethics is crucial
A documentary recently revealed that data traders often neglect ethics. Much of our online surfing habits and smartphone usage eventually ends up in a database, often completely unnoticed, sometimes associated with your IP address, or even your identity. An accurate profile can be built up within a matter of a few weeks. Dutch companies also use these profiles with illegally obtained data to better profile their customers and prospects.
But this doesn’t mean that data collected otherwise can always withstand scrutiny. All in all, a major reason (even more so than before) for an organization to develop ethical guidelines for the use of (externally) obtained data (legally collected) and to minimize the risks of personal data leaks. In order to not give dishonest traders a chance.
“Criminals exploit illegally obtained data at lightning speed: we are not yet completely prepared for large-scale data abuse. I wouldn’t be surprised if people become reluctant to submit their data, which may result in impeding the Business Analytics process.”
Mr. Leo Kerklaan
2. Big Data is about to break through into the public sector
When it comes to Big Data, the public sector still has much to gain. The sector has subtle goals and good information is therefore required, with more details and implications than traditional BI environments might offer. For the private sector, the driving force is mainly the improvement of profit and sales. Once the non-profit sector knows how to translate this focus to their own situation, successful Big Data initiatives are just around the corner. These initiatives can substantially improve service delivery to citizens. The current Big Data economic value for the public sector (worldwide) is estimated at grossly 4.2 trillion EUR, much more than applications in other fields (source: Cisco).
“Thanks to advanced technologies like smart sidewalks, camera systems, and intelligent street lights, known as Smart Cities, there is so much more data available. Public sector organizations are aware that they need to take this on together and this will really break through in 2016”.
Mr. Minne van der Sluis
3. Internet of Everything
Capturing this Big Data requires an enabler, which is the Internet of Things (IoT). Take, for instance, a portion of a highway: the difference between 10 years ago and today is vast. The cars are intelligent and interconnected, just like the drivers via their smartphone or wearable technology. But there’s much more to it: road sections are packed with measurement and control technologies, while cameras monitor road users and road use. Cities and public outdoor areas will soon be filled with all kinds of smart sensors. A cow now produces data (200MB per year) as well as milk. All these sensors collect data, provided with the necessary safeguards and integrated with each other, that enables smarter working and lifts intelligent management to a higher level.
“Everything is becoming digital and interconnected; even more than we could imagine a short time ago. The further development and implementation of Data Science will help progress even more.”
Mr. Minne van der Sluis
4. Finally, a shared version of the truth
In 2015, Business Intelligence became more democratized and made available to more people thanks to the progressive adoption of cloud, mobile, and self-service BI technologies. The fact that users are far more able to go their own way stands in sharp contrast to the past situation, where definitions were tightened, and we tried to force a version of the truth with a boarded-shut data warehouse, among other things. This did not work for many organizations, and they were not always successful.
The coming year will provide access to BI for more business users, which will also be made more comprehensible: self-service business intelligence allows employees to work out their own insights and apply their own definitions. This gives traditional BI managers headaches because the figures all have to be correct. Fortunately, help comes from an unexpected source:
“With more self-service BI, end users can produce even more nonsense than before. Fortunately, we can see that the social component of BI has been booming simultaneously. Using elements similar to LinkedIn, users can initiate an underlying dialogue about the numbers, start discussions, and keep each other on their toes. The combination of self-service BI and social BI may be the missing link to the long awaited holy grail: the shared version of the truth and, simultaneously, being successful with BI & Analytics”.
Daan van Beek
5. Performance remains a problem
Computers are becoming increasingly more powerful with more and more memory, while the cloud technology innovations allow for an easier connectivity. This results in the progressively easier processing of large amounts of data. At the same time, there is more available data that is also more complex than before, because companies can get more and more value from their data, which they then seek to exploit; thus, performance remains a huge problem. Intel develops a much faster processor, and Microsoft uses it to its fullest. The BI environment runs on cloud-based supercomputers, and the business wants to analyze increasingly large and complex problems that require even more robust computers.
“Almost every week, I hear someone complain that their Business Intelligence software is barely moving in their company. Despite the booming in-memory analytics and greater processing power, more ambition in the area of BI involves persistent performance problems. A race for computing power that will constantly continue.”
Daan van Beek
6. Data lakes are draining data warehouses
In 2016, traditional data warehouses will be increasingly absorbed by data lakes, large storage locations for big data. This is a logical trend because data lakes offer many advantages over traditional types of data storage. A comparison between a bottle of filtered water and a lake is a good example. The bottled water is intended for just one application; the lake water allows multiple applications because it is not filtered. In today’s turbulent times, this may result in a great competitive advantage for an organization because of greater agility. In addition, the data lakes are less exposed to hardware failures because solutions like Apache Spark and Hadoop take that all into account.
“But beware: the lake may freeze or turn into a data swamp. In addition to data quality issues, data lakes still have to overcome several challenges. Security and integration are such challenges, among others. Nevertheless, you can see that many companies are trying to transfer much of their data from traditional data warehouses to data lakes. They keep only data that is necessary for compliance, consolidation, and reporting. The rest is transferred to solutions like Hadoop and Spark.”
Daan van Beek
7. Algorithms will make the difference
Data is either available in abundance, or is made quickly and easily accessible. The difference in the coming year will not be with more or better data; instead, it will be the unleashing of new algorithms and techniques. This means that the demand for good data scientists, capable of building such algorithms or choosing the right technology, will only increase. Colleges and universities are rapidly filling up with such people, but a substantial shortfall is still expected for 2016.
“The really big companies don’t have a problem with finding someone who will do interesting things with their data full time. But (larger) SMEs need to join forces and involve the customer or other interested parties. Together, we can consider crowd sourcing and how to use your knowledge, expertise, and data in a smart way.”
Mr. Minne van der Sluis