If you want proof that big data is a big thing, just look at the jobs pages. Inside, you'll find desperate appeals for statistics-savvy data scientists, and the salaries are often eye-watering.
They're evidence that businesses in the UK and beyond are looking at ways of using the data they already collect – whether that's about customers, their industry or about financial markets – with as much efficiency as possible.
Digital markers
Data is everywhere, it's doubling every two years, and it's fast becoming a business' biggest resource. The digital markers we all create – whether a GPS-tagged photo, check-in on Facebook or a click on a website – could last forever.
So far, we've got about 1.8 zettabytes of data on planet Earth, but it's how we link up, analyse and use all of this data – and much more besides – that's going to completely change our world. And that means business.
Some say that by mastering big data, companies can stop wasting time on useless initiatives and pitches that go nowhere - and increase margins by a whopping 60 per cent.
Even if it's a fraction of that figure, harnessing big data properly can make the difference between a profitable business and a non-starter.
Big data is big money. ABI Research states that global spending on big data by organisations exceeded US$31 billion in 2013, and will reach a staggering US$114 billion in 2018 – and that includes salaries as well as hardware and software.
It's a data-rich, skills-poor environment at present, but the future will be one of increasingly sophisticated hardware and ever-better data analytics that collect, calculate, predict and much more besides.
The iceberg problem
"One of the companies we're working with has 6,670 'icebergs'"
Unfortunately, big data is scattered. It's estimated that IT departments spend about 70 to 80 per cent of their time maintaining existing systems and databases, largely because there are so many of them.
Separate, created for narrow purposes, and often based on databases that don't talk to each other, it's what's known as 'the iceberg problem', where systems can't exchange big data, which therefore goes untapped.
"One of the companies we're working with has 6,670 'icebergs' used by various arms of the business," says Dr Andrew Sutherland, Senior Vice President of Technology at Oracle EMEA.
"Some are very important, each one was built for a reason, and all had and still have a business case; and they were all built by very clever, experienced people – but each application has been built to fulfil one need, and they use completely different databases."
False economy
It's a common problem, but businesses who've invested a lot in such icebergs often don't want to replace them. For a business wanting to embrace big data, that viewpoint represents a false economy.
Petabytes of big data are now flooding in to businesses and a consistent, common platform is required, capable of exchanging data and running apps – whatever their source – using only one stack that can be managed, maintained, recovered and backed up as one.
It's all about consolidation. "Whether you're a police force, a gas extractor or a retailer, there will be areas of your business that you want to be able to do more efficiently or cleverly – and you can only do that if you consolidate and simplify your IT, so you can concentrate on the apps that add value to your business," says Sutherland.
One problem here is that there often exist huge teams of IT consultants whose job it is to specify and commission more icebergs; asking them to give their blessing to using one unified, interoperable system that best takes advantage of big data is like asking turkeys to vote for Christmas.
But if you do get big data into one place, what happens to it next?
