Machine learning & super-computers
The big data revolution will be led by machine learning and super-computers. They've been called the 'coal in the furnace' of the internet that will drive Web 3.0, but the sole reason for the new generation of artificially intelligent super-computers is the massive rise in personal and business data.
Eventually enabling websites and apps to track and predict our every action and desire, super-computers feed off big data.
Reasoning, perception, social and even some degree of cultural awareness are the selling points for Watson, IBM's super-computer that runs on 16 terabytes of RAM, and brings a much-needed injection of artificial intelligence to big data analytics.
Its smart learning DeepQA software is able to search databases, spot patterns and make predictions, which could revolutionise the financial and medical industries, both of which are currently drowning in data.
Big data in medicine
As well as reducing the need for investment bankers, big data analytics could also banish bad decisions by doctors.
"Watson could be used to search through millions of pages of academic research and drug trials, something a human could never do or keep up with, and in double-quick time," says Joe Peppard, a professor at the European School of Management and Technology in Berlin, Germany, which consults on IT strategy for large corporations.
"It could provide a doctor or nurse with the required diagnosis and even the most appropriate medication plan, having taken into consideration all the latest thinking on any specific health problem."
Call centres streamlined
"Watson is able to search a far greater amount of data in a much faster time"
Such data analysis could also be used to vastly improve call centres, where about 60 per cent of enquiries end in frustration for callers.
"Watson is able to search a far greater amount of data in a much faster time," says Peppard, "so in many cases it could pre-empt enquiries so that the answer is available before the question has been asked."
"Machine learning, and its application in advanced analytics, is one area that will make both the public and private sectors data-savvier than anything we've seen so far," says Dan Shey, Practice Director at ABI Research.
"Big players such as IBM and HP are understandably moving to this direction, but at the same time we can also see analytics startups, like Ayasdi and Skytree, that have machine learning in their very DNA. Eventually, such innovations will put analytics within any domain expert's reach. At that point, data will stop being big."
Uncovering the Universe
Harnessing big data properly can mean new discoveries and possibilities, and nowhere more so than in deep space.
The European Space Agency's Gaia mission to create a complete three-dimensional map of the Milky Way involves taking super hi-res photos of the heavens, but to make any use of this data requires a powerful information system.
Archiving and processing scientific data collected by the Gaia satellite, the Italian National Institute of Astrophysics (INAF) uses a data management system from Oracle to store its "very precious heritage of astronomical data that will have to be stored for the whole 21st century and beyond," says Roberto Morbidelli, Scientific Operation Manager at INAF.
"In five or 10 years' time, big data will be a necessary part of normal business"
Processing petabytes of big data from Gaia is a huge task, but Oracle has history; it also works with the operator of the Large Hadron Collider, CERN.
The sources and uses of big data in business are now being revealed, but if we know one thing it's that the hyper-efficient collection and analysis of data is already becoming a major tool.
"In five or 10 years' time, big data will not be a source of competitive advantage, it will just be a necessary part of normal business," says Eddie Short, UK and EMEA Leader for Data and Analytics at KPMG Management Consulting. "Without exploiting it, a business will die."
