Data science was once dubbed the “sexiest job of the 21st century”. Yet, when you dissect the working day of a data scientist, it’s hard to see why. Results from a recent IDC study found that, on average, data professionals spend 67% of their time searching for and preparing data for analytics. When we put that into the context of an average working day in the UK, that’s almost five hours which is spent on data admin alone.
Jean-Michel Franco, Senior Director Data Governance, Talend.
The same research also found that only 12% of data workers’ time is being spent on delivering insights, despite being critical to supporting business amidst the current crisis. Again, when we put this into context, that’s just over one hour a day. This is a tiny amount of time that is being spent on, arguably, the data professional’s most important job in this current climate – creating meaningful, actionable insights from the data that is being generated within the organisation.
Monopolizing data professionals' time as data volume grows
Sadly, if businesses continue on this trajectory during this pandemic, the monopolization of data professionals’ time will only get worse as the volume of data grows. More data means more siloed data – as teams struggle to understand what data is being generated and where it’s being stored.
More siloed data means incomplete data sets, reducing the quality and impact of the data being collected. The result: data chaos. A chaos that the data professional will find themselves responsible for and, perhaps, have to spend even more of their time sorting out.
This is a problem only amplified by the effects of COVID-19, because of which means data is coming from an increasing number of sources both within a business and external, creating more silos as businesses fight to understand what the pandemic means for them and how they should respond to best serve their customers, employees and society at large.
Getting access to accurate and trustworthy data are more needed than ever in times like these. Are we getting control of the pandemic, are we at risk to fall short of health resources, are we reaching the plateau, can geolocation data help stop COVID-19 and how can we manage it to control the effect on our privacy? Now we all understand how important data is to help us answer those crucial question and ultimately win against the crisis.
We also envision how hard it can be to collect this data in real time and create the ecosystem of skills and trust to turn it into outcomes.
Why complete data has become so important – moving from a fragmented to a holistic view
Like water, data now plays a vital role in our everyday lives – supporting societal wellbeing, economic growth and can even be transformed into green energy. But when it is out control, leaked or storied in excess, data can lead to massive problems and cause mammoth damages.
In fact, a partial or biased view of data can result in a negative impact on the bottom line or, in some cases, lead to discrimination. Only with a full and mastered view can data professionals deliver accurate, actionable insights that will provide the business with clarity to make the right decisions and the confidence to drive them forward in an uncertain, risky climate.
For this very reason, data is the lifeblood of digital transformation and a key asset in a crisis. And so, when it comes to evolving digitally and helping support organisations through the challenges that we are facing with this new crisis, businesses of all sizes should not only get access to all of their data but also to weave them together and ensure they are trustworthy.
Yet, what many businesses don’t know is that this is exactly the position they are currently in – analyzing incomplete data sets to inform strategic business decisions.
Data intelligence – the missed opportunity
The severe data talent shortage that the world is currently facing is only further monopolizing the data professional’s time. The Royal Society found that demand for data workers in the UK has tripled over the last five years as an increasing number of businesses want to take full advantage of the benefits of data. Figures from the European Commission add to this, suggesting that there were 5.7 million data professionals in 2018, growing to 10.9 million in 2025. However, this is still way behind demand. Without a full and developed team of data professionals, data initiatives will fall to a limited number of individuals capable of carrying out the tasks – stretching their time even further.
Not only do companies need to attract the resources and up-skill their workforce, but they also need to find ways to make their current team deliver more. Yet, with so much time being spent on data admin, key activities including data analysis and data strategy are being vastly neglected. This can, and has already for some, translate into a huge productivity crisis, resulting in lower revenues, less innovation, a poor customer experience, and higher costs and risk. Something most businesses cannot afford at the best of times and will want to avoid as the crisis continues.
This means that organisations have to double down their efforts to make their data more meaningful. This all about data intelligence, which IDC defines as the ability to “leverage business, technical, relational and operational metadata to provide transparency of data profiles, classification, quality, location, lineage and context; enabling people, processes and technology with trustworthy and reliable data”. By prioritizing data intelligence, data professionals will no longer have to sacrifice hours finding, sorting and preparing data from across the business. Data professionals will be able to look at business data as a reliable source from the initial point and focus their efforts into delivering key business insights.
Data chaos to data clarity – without data admin
Instead of wasting the scarce and precious time of data professionals on data admin, businesses should invest in collaborative practices and tools that accelerate the data-to-value journey with easy and searchable dataset documentation, quality proofing and promotion. Deploying these tools effectively will mean data is pinpointed across data sources, even those that are siloed, and turned into reusable and shareable data assets with a single point of governance and access. This will help to free up the valuable time of data professionals and allow them to work on analyzing the data and delivering actionable insights to direct business strategy.
For data professionals, initial data analysis should be as easy as that of the experiences of the digital consumer. When they book a meal delivery, order essential groceries, or purchase loungewear online, not only are they directed to the best offer in a few clicks, but easy to consume ratings give an assessment of quality and often dictate a purchase. Data professionals should be provided with clear and concise initial insights that help them direct deeper analysis and drive strategy.
Therefore, when deciding which tools to invest in, businesses should look for those that deliver and that work a little bit harder for data professionals. They should not only accelerate access to data but also provide the data professional with a speedy method to assess data relevance and trustworthiness at first sight. Business applications that are able to provide an instant assessment of data health and accuracy, based on data quality, data popularity and user-defined ratings, greatly reduce the searching and preparing data professionals have become accustomed to. This will allow data professionals to focus their efforts on leveraging the power of data intelligence for the organisation.
In today’s climate, data is shaping up to be one of the most valuable commodities in order to help support businesses through the pandemic to lessen its worst effects. However, many businesses are failing to take advantage of the full extent of its power at this most critical time. Data professionals are bogged down with data admin meaning there is little time left for analysis to support data-based business strategy to help organisations position themselves to accelerate out of this pandemic when it eventually subsides.
Investing in the right tools will help existing data professionals address efficiency and productivity problems from the outset. Businesses should not settle for imperfect data and if the correct tools are deployed, firms of all sizes should have access to quick and trustworthy data that their employees can leverage effectively.
- We've also featured the best video conferencing software for improved productivity.
