The high costs of storing data locally in a cloud native era

With data breaches looming, we’re seeing organisations being doubly cautious, opting to keep their data storage IT assets onsite. In our recent research report, High Cost of Cluttered Data Centers, 600 data center experts cited their top priorities as security, automation, productivity and cutting costs. Security really leads the agenda, and while the cynical amongst us might think that is due to the threat of fines, it turns out that what is really focusing minds is the threat of reputational damage and the knock-on effect on company valuations.

With everything that we see in the news about a universal move to cloud, you might expect that on-premise data storage will soon be coming to an end. In fact, we’re likely to see onsite continue long into the future. This is mainly due to the fact that organisations like having control. 

There are plenty of CIOs who feel they simply can’t leave the uptime equation to someone else—or relinquish control over their data. This is not to say that we aren’t seeing massive growth in the uptake of cloud services, and I have no doubt this will grow even faster in the coming year. In my view, the most sensible way forward for organisations is a hybrid strategy that combines the benefits of cloud and the security of on-premises.  

The main challenge facing organisations that opt to store massive amounts of data is cost. Findings from our study of 600 data center experts identified that two out of every five organisations that store data in-house spend over $100,000 every year on storing hardware that could be redeployed, sold for reuse or returned to the manufacturer for a refund, and that also represents a security and compliance risk.

This challenge is wholly avoidable, but in light of regulatory pressures, organisations have started hoarding assets containing data in-house for fear of failure to achieve compliance with data privacy standards. 

The reality is that stockpiling hardware is counterintuitive and offers a very short-term solution. As organisations continue to store more data in-house, the surface area for attack gets bigger and it becomes increasingly difficult for them to manage. What organisations need to do is manage both data and asset lifecycles by employing a process to regularly review and assess the value of that data and make a decision on whether it should be archived for future use, or sanitised. 

There are many drivers for cost tied to storing large amounts of data. Paying for the space to store hard drives holding data is one of the main causes, but it is by no means the only one. Costs associated with noncompliance also have a big impact, as does locking in the value of equipment that could be resold or returned, and the increased risk of breaches from hoarding assets containing data that could lead to substantial financial damage . 

For example fines of up to $1.5 million can be issued for HIPAA violations of keeping data past its retention date. Our research showed that 54% of companies have been cited at least one or two times in the past 24 months by a regulatory or governing body for failure to comply with regulatory laws such as GDPR. Return Material Authorisation (RMA) hard drives should be returned, under warranty, to the manufacturer for a replacement or refund. However, organisations are keeping hold of them, along with the sensitive data stored on them. As such, they are incurring additional costs from penalty fees for “keep my disk” clauses. 

These fines, borne out of compliance failure and a lack of best practice process, can potentially be crippling for organisations, yet they are easily avoidable. It’s key that organisations start investing in tools and software and create policies to meet with new standards.  

Typically, hard drives can incur faults at some point during their asset lifecycle. Degradation might occur from heavy use, or issues relating to dust or overheating. We are seeing data centers invest in far more efficient, state-of-the-art cooling methods, which significantly improves hard drive health, but this does not make them faultless. The nature of things means that you will always see some drive failures, but the life span of drives is improving constantly. 

In the instance that you encounter hard drive degradation, organisations need to have software-based processes in place to erase and return drives since many hard drives still respond to read and write, even if internal diagnostics and warnings advise complete replacement. If a drive is completely broken, then a fall-back physical destruction process is necessary as it can guarantee data sanitisation, material should then be recycled to reduce the environmental impact.  

We’re seeing massive uptake in the adoption of cloud, so we’ll certainly see businesses move away from storing data in house. The real cost issues with storing data locally is hoarding useless hardware, which as we’ve discussed can be easily avoided through proper data sanitisation and asset and data lifecycle management.

Moving to cloud is not without its disadvantages, and there are a number of things organisations should consider ahead of cloud migration, such as a cloud exit strategy and implementing a data audit trail to guarantee data can be accounted for. Again, the most sensible way forward is a hybrid strategy that encompasses the benefits of cloud and on-prem. 

We’re going to see a far greater focus over the coming years on data center energy efficiency. It’s an issue that has been looming ominously over the industry for some time now—with the  data center expected to account for as much as 20% of the world’s electricity by 2025. The optimal use of existing data centers is what the industry should be working towards. Data lifecycle management and proper sanitisation in lieu of physical destruction, which causes yet more concerns about e-waste, is the type of progression that we need to see in the industry. 

The physical security of data centers will also become much more of a hygiene factor with a great focus placed on data security and an ability to provide it on a number of different dimensions. As we continue to experience an explosion of data, the surface area for a potential attack grows considerably. In addition, tighter regulations around how organisations handle data will make securing data through proper data lifecycle management, erasure and encryption best practices all that more important. Additionally, we’re going to see data centers forming regional clusters or data center hotspots. Scandinavia and Northern Europe have already become very strong clusters, and we’ll likely see greater proliferation of these clusters in North America and Asia. Easy access to natural cooling, renewable low-cost energy as well as security, stability, connectivity and know-how will drive the quickest growth. 

The way organisations store data is set to change because it needs to. New laws are pushing organisations to up their game in terms of best practice and we are going to continue to see the instatement of new regulation on an international level. This has to happen, because if it doesn’t, there will be huge consequences as a result of mishandled data.