The 'black box' approach
If you want genuinely useful business insight, you have to work at it. "Too many think an analytics strategy means choosing a specific black box to be fed with specific data feeds, and applying specific pre-set algorithms," says Nick Clarke, Head of Analytics at international analytics, software and consulting services company Tessella. "But many problems are too complex and too subtle to automate in this way … the correlations that pop out the other end are not magically wrapped up in valuable business insight."
All big data is not the same, and not even Google can get insight from any unstructured data set. See the failure of the Google Flu Trends (GFT) project for proof of that – complex issues require sophistication when it comes to big data.
Getting it right
However, there are plenty of examples of getting it right. "The pharmaceuticals industry has long used vast complex data sets to identify profitable areas for new research," says Clarke. "Although still far from perfect, their maturing combination of data analytics and clarity of vision of what they want to achieve, puts them at the front of the game."
Their secret is the embedding of scientifically literate data experts within specialist research groups. "Human expertise is vital when framing the problem, contextualising the insight, and uncovering bias, both in the data feeds and in our assumptions," says Clarke. "The data strategies of too many organisations fail at this level."
On the cusp?
Some think we're now on the cusp of being able to deliver insights on a computational scale. "The information age is finally finding its 'steam engine'," says Pohlmann. "This alignment of scale and technology means that we can now create algorithms to support things like machine learning processes, which sees big data evolve from being purely a numbers game to an integral part of any corporate planning process."
Is big data a big failure so far?
"No," says Pohlmann, who thinks that the problem is that many organisations are wedded to just their existing Enterprise Data Warehouse (EDW), usually in silos, and practicing business intelligence techniques that fall far short of the proactive, predictive operational analytics that can actually deliver on intelligent decision making.
"With the continuing rise of IoT, AI and machine learning, there's going to be more pressure than ever applied to organisations that are stuck in the past to adopt new processes and techniques," he says, "and actually start to benefit from what big data has to offer."
The pace of change
Benefitting from big data can take time, but companies paying only lip service to big data are putting everything at risk. "As companies gather more and more granular data on what they do, the potential to gain understanding and plan accordingly is not just a profitable undertaking, it is a necessity," says Petter.
"Transformation is being forced on organisations at an ever-increasing pace. They must adapt to new ways of doing business, new markets and new practices." Or what? "Or die," says Petter.
