Few technology terms have ever been hyped as much as big data has over the past year or so. But amid all the talk about the business advantages that big data could bring, it's been forgotten that many businesses have not yet taken practical steps to roll out a big data strategy.
In a recent survey we ran at Talend, only one in ten of our sample said they were actually engaged in large scale roll-out. That was up from just 2% a year earlier but still disappointing.
The truth is the big data market is still in its infancy today. We have seen a lot of interest out there in the market – and that interest is definitely growing. So what's the problem? What has held businesses back so far? And are we seeing signs of a change?
The constraints
Two of the top three constraints we identified in our survey were budgetary restrictions and skills shortages, both widely recognised as key barriers to any IT endeavour.
The potential scale of most big data projects explains the financial concerns. Also, skills are key because big data requires people to be able to integrate any number of large and inflexible disparate data sources and skilled data scientists to analyse the collective data streams.
You could argue these barriers are more about perception than reality. The truth is, big data projects do not have to come with a massive price-tag.
Being able to run open source databases and integration tools over an open-source Hadoop platform allows big data integration and analysis applications to be run cost-effectively across commodity server clusters, thereby reducing hardware spend. And the latest advances in technology are helping to close the skills gap.
Analyst research firm, Gartner defines big data as "a combination of high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making." And this innovation is key in helping to hide the underlying complexity of big data from developers and users alike.
The winds of change
Last year, we saw signs that big data was starting to happen. It was a year of experimentation, a year where many organisations were still carrying out proof-of-concepts. It was still an embryonic technology.
Very few organisations had moved outside the sandbox and started delivering productive use cases – and even fewer had any return on investment or tangible results to refer to, but the promise was there.
In 2014, we are already seeing more companies 'getting their feet wet' with big data but we will also increasingly see big data reaping the rewards of the trials of 2013, crossing the chasm and entering into mainstream adoption.
New technologies are coming on stream that enable real-time and operational big data; technology platforms are available that are helping organisations' big data investments to scale.
And so, a growing number of companies are going live with big data projects and starting to get results from their big data implementations. Big data may still be in its infancy today, but this is the year it will start coming of age.
- Yves de Montcheuil is the Vice President of Marketing at open source integration specialist, Talend.
