In December of 2022, I participated in an eWeekChat event talking about predictions for 2023. One of the predictions I made was that “new flash capacity will get within striking distance [of the cost] of new HDD capacity in 2023.” With 2023 underway, I thought it would make sense to explain some of my reasoning and why we here at Pure Storage are so confident that the disk/flash TCO cutover is closer than many people may realize.
Learn how all-flash is replacing HDD in the data center.
It’s important to note up front that flash has already been the obvious choice for any performance- or latency-sensitive workload for some time now. In fact, in the data center of 2023, the idea of “performance disk” is mostly an oxymoron. Since flash memory’s entrance into the enterprise storage arena, it’s been knocking down hard disk tiers one by one, from top to bottom, until now there’s really only one tier left where disk drives maintain a market share advantage: nearline, capacity-focused drives. This is the lowest tier of online data storage, but it also represents a huge portion of bytes being stored, as well as the growing majority (more than 70%) of hard drive capacity shipped each year. That’s the tier we’re talking about today.
“For the last 15 years, it has always been the next [hard drive] tier to fall, but now there is only one tier left, and only one reason left to use disk, and that’s cost. With this crossover, it will be the last tier of drives to fall.” – Pete Kirkpatrick, VP of Hardware Engineering, Pure Storage
As NAND price-per-bit continues to decline—at a rate much faster than nearline hard drives—that, combined with the power, space, and cooling savings, higher performance (yes, there’s a cost impact of performance as I’ll explain!), and better reliability will soon render hard drives the less cost-effective option when taking all these elements into consideration. To understand why, let’s break down each of these cost components.
Retail flash price in orange, HDD in green. Data plotted from https://jcmit.net/diskprice.htm
Media Cost
The first-order cost is always going to be the media itself. While today the cost-per-bit of a hard drive is still lower than the densest flash, there are several key trends to pay attention to in 2023.
First, all the major tier 1 flash manufacturers have demonstrated significant density increases that are planning to come to market this year, over 200 layers of stacked 3D NAND in some cases. This increase in density will translate into better cost efficiency, as well as have knock-on effects on power and space savings.
Second, if analyst predictions hold, NAND prices overall are expected to decline through most of 2023, continuing a trend that began in late 2022. In fact, were it not for a major flash manufacturing contamination event that kept prices elevated in February of last year, we probably would have seen more decline in much of 2022. That event delayed price drops which we’re starting to see materialize now, but at an accelerated rate. Combine that with other economic headwinds, and some are predicting the cost-per-bit of NAND to continue to decline in 2023.
Lastly, while this probably won’t be felt in 2023, penta-level cell (PLC) technology was first demonstrated more than two years ago, and the first working PLC drive was shown at Flash Memory Summit in August of last year. Moving from four to five bits per cell would improve density by 25%, which presents an obvious cost-efficiency advantage. Pure Storage was the first to ship a QLC-based enterprise storage array in 2019 with our FlashArray//C™ product, and now ships in our FlashBlade//S™ product, as well. Pure Storage’s DirectFlash® technology allows us to adopt new NAND technologies efficiently once they’re viable, in addition to allowing us to build larger drives with more flash but less other components (like DRAM) than commodity SSDs.
Energy Cost
Data center infrastructure represented over 1% of global energy consumption in 2021, and this share is growing. As a result of rising energy costs, energy consumption by information technology has become not just a concern from an environmental standpoint but now also a significant economic one.
This means that when looking at total cost of ownership, energy consumption will play an even more outsized role in 2023 and beyond, and infrastructure that is more power efficient will have an even greater economic advantage over the competition. As flash density continues to increase, power efficiency will surpass hard drives and thus contribute to a lower effective cost.
In 2022, Pure Storage published our first ESG report which compared our systems to competitive all-flash alternatives, where we showed that our systems can use up to 84% less energy. Pure’s energy efficiency will be a leading factor in enabling Pure Storage products to achieve a lower TCO than disk in the future.
Higher Performance
While it may seem counterintuitive to talk about performance as it relates to cost, it does have a significant impact on the effective cost efficiency of a data storage platform. In two key ways, the flash performance advantage over disk can translate into cost advantages.
First, because disk performance is still so low, even in workloads with modest performance requirements, you can find yourself with stranded capacity. Yes, your spinning drives may hold 20TB, but if the performance of the system taps out and they’re all only 16TB full, you aren’t getting as effective a cost-per-bit. Flash doesn’t have nearly as much trouble in this regard, especially because read performance stays predictable even as drives fill up, unlike hard drives. Remember short-stroking?
Second, any storage environment that focuses on resiliency has some kind of redundancy strategy. That may be storing multiple copies of data, using mathematical formulas such as RAID or erasure coding to store parity data, or both, in addition to keeping spare capacity around in the case of a failure. These requirements are driven by statistical realities that nothing is perfectly reliable and that systems have to automatically restore resilience as quickly as possible before another failure occurs.
This means that the performance of any one device is a variable in the equation used to determine how much redundancy you need to build into your system. Many years ago, hard drives had gotten large enough (and were still slow enough) that after a drive failed, rebuilds would take so long that math dictated adoption of dual parity strategies, and today that can be triple parity or more. With flash devices, faster rebuild times mean less bits need to be dedicated to resiliency structures, which equates to better effective cost efficiency. Pure Storage systems offer robust, automatic resiliency tuning that takes all these aspects into consideration, without requiring users to take a college-level statistics course to determine what RAID level they need to set.
And this gets even better when you take into consideration the last component:
Better Reliability
Our data shows that our annualized failure rates (AFR) are significantly lower than the industry benchmarks for spinning disk, as well as lower than industry SSD averages, to boot. For a good explanation of why, check out my Better Science, Volume 1 article. If drives are more reliable, this goes into the same resiliency considerations as above.
But beyond that, it also means that more reliable devices need fewer replacements (obviously), and replacements are not free. While most customers with support contracts don’t feel the cost directly, the cost of this unreliability is baked into the cost of those very support contracts. This is one reason why traditional storage vendors’ support costs would skyrocket in years 4, 5, and beyond—this was the time horizon on which spinning disk reliability takes a nosedive.
This is one element that makes Flat and Fair maintenance from Pure Storage possible—our flash drives are more reliable, with longer usable lifespans, and therefore we don’t need to hold customers hostage with outrageous maintenance costs to incentivize them to upgrade. It’s why 97% of Pure Storage arrays purchased six years ago with an Evergreen™ subscription are still in service today.
Flash Is Just Better
Let’s not beat around the bush: No one buys spinning drives because they’re better, only because they’re cheaper. In almost every way, flash memory is superior to spinning magnetic storage, and where flash is already better, those advantages are increasing. Where flash is still behind disk is cost, and the road to the disk/flash crossover point is getting shorter and shorter—and Pure Storage is at the front of the convoy. While it may not be the most ambitious crossover event in history, it will certainly be one of the most consequential for data centers, for the IT industry, and for the world.
Agree or disagree? How close do you think the crossover is? Continue the conversation with me on LinkedIn or Twitter.
Read more about the author, Justin Emerson
