It was once impossible to imagine that vinyl records would no longer exist, nevermind their successor, the CD. Likewise, we’ve witnessed the dramatic rise and fall of the VHS cassette, DVD, and their iconic retail outlet, Blockbuster. All of these once popular hardware innovations, and their delivery methods, are now a thing of the past, replaced by digital music and video streaming, delivered via the likes of Spotify and Netflix.
What can we learn from this in 2023? Perhaps now we can also envisage a world without spinning hard disk drives (HDD), and even contemplate the limits of off-the-shelf flash drives. Like CDs and DVDs, a mere decade ago this would have seemed implausible. However, if we look at storage use case trends and drive technology advances, it’s not so far off that HDDs will disappear, with small capacity flash drives not far behind.
That’s because new ground has been broken in solid state drive (SSD) capacity and density, and therefore cost-per-GB. So, while SSD has long been mainstream for high performance use cases, the prospect is that very dense flash modules will supersede disk in areas where the technology has been hanging on, such as nearline storage, cold archive and hyperscaler deployments.
There is a growing need for increased storage capacity coupled with rapid access across all workloads. This is driven by heightened prevalence of analytics and AI, with generative AI the latest wave to drive up storage capacity needs.
The current marketplace is shifting because of technology advances in which some flash storage makers have gone well beyond the off-the-shelf SSDs that top out at about 22TB. Instead, some can put many more flash chips on a drive module to achieve capacities of 75TB and more, with hundreds of TB promised in a year or two.
The result is a claimed current total cost of ownership (TCO) price-per-GB of 20¢. Soon that will decrease to levels where spinning disk is simply no longer competitive.
Wes van den Berg is VP and General Manager of UK&I at Pure Storage.
Flash modules: higher capacity, more efficient, less power-hungry
Some have predicted that spinning disk will be obsolete before the end of the decade. With flash modules already at 75TB and such low $/GB prices that means HDD will likely be driven out from use cases where it has thus far held on.
These include nearline use cases, such as file and object storage for unstructured data that have gained prominence as organizations want rapid access for analytics and AI/ML processing.
Also, as flash module drive sizes achieve capacities of hundreds of TB there is the prospect it will become compelling to switch from HDD for further secondary storage use cases currently reserved for HDD.
This is before we address the huge savings flash delivers in energy consumption and maintenance. This is important in the context of current high energy costs and the drive for organizations to reduce energy consumption and achieve ESG goals.
Not only this, but with the amount of TB it is possible to pack into flash modules, data centers can reduce rack space from hundreds of U to just a few for petabyte capacity. At the same time there is much reduced energy consumption as flash doesn’t need power to keep disk platters and heads moving. In fact, a shift to utilization of modern flash technology in data centers could reduce power consumption by as much as 80%. Finally there is the bonus that flash drive modules are much less likely to fail than HDD equipment full of moving parts.
All this also means that off-the-shelf flash drives are threatened too. That’s because the big drive makers offer SSD capacity in disk-like form factors that often use connection methods developed for spinning disk. Critically, they are also limited by a lack of software that can knit together many more flash chips into larger capacity modules.
That kind of controller intelligence is the preserve of array specialists with experience of architecting storage operating environments and is likely to remain so for some time to come.
Gain advantage from HDD’s extinction event
So, what should customers do to prepare for the potential extinction of the spinning disk HDD and the eclipse of off-the-shelf flash?
Companies need to plan for the all-flash data center, meaning all workloads moving to flash storage within the next five years. The flash modules that become available – running up to 100s of TB – will allow storage arrays to scale to petabytes in a few U of rackspace. That will enable flash module storage to support workloads that go deep into secondary use case territory, such as backups, data warehouses and data lakes, archive and mass hyperscaler storage. Much-reduced power, space and headcount needs will bring organizational savings for the business, in which IT can be the catalyst.
To achieve this, businesses should evaluate vendors that have a roadmap for high density flash module storage capacity that explicitly targets a range of workloads from the most performance-hungry to those categorized as secondary.
At the same time, organizations should evaluate vendor purchasing options, from traditional outright CapEx with a subscription to upgrades, to purely OpEx storage as-a-service. A clear strategy here prevents lock-in and stagnant hardware further down the line, allowing customers to build in smooth capacity and technology upgrades for years to come.
With the forthcoming disappearance of a storage media that’s been around for decades, customers need to plan ahead so they are not caught out by a sudden VHS / Blockbuster / vinyl-like obsolescence in critical areas of their data center.
