One of the common questions that we hear from customers is how should I test the performance of Flash AFAs. A simple question… right? Well, it turns out there are diverse set of opinions out there and (as an industry) the answer is clear as mud.
I recently had the pleasure of sitting down with Eric Burgener, Research Director at IDC and author of IDC All Flash Array Performance Testing report to record a video on this topic. You can view the video here.
Eric and I had a candid discussion and explored:
– Why is the transition from disk to AFAs accelerating?
– How should customers test AFAs?
– Should they conduct Real-world or Synthetic testing?
– Why AFAs are different and the implications for AFA testing?
While you can watch and hear the full details, I’ve captured key highlights below.
What’s your objective?
Start by asking what you’re trying to achieve. You want to get an accurate assessment of how the AFA will perform with your production workloads during normal and failure conditions. Another words, determine what the AFA can do for your business.
How should you go about testing the AFA to achieve your objective?
In short, you need to get real! Real-world testing is the best and most accurate approach for assessing how the AFA will perform with your production workloads.
IDC paper outlines Class A-E test types:
– Class A: Production workload testing
– Class B: Workloads generated by app-specific tools
– Class C: Workloads generated by general purpose workload generators
– Class D: Intelligent performance corners testing
– Class E: Classic performance testing (aka Hero testing)
Class A (aka “real-world”) testing is recommended. If it’s not feasible to conduct Class A testing, then the next best thing is to model your production workloads as closely as possible with Class B or C test types. The next best alternative is intelligent performance corners testing (Class D).
Can Synthetic testing inadvertently mislead you?
Legacy testing centered around Class E is misleading. Today’s workloads differ significantly from classic “hero test” workloads (100% read, fixed 4K/8K block size, etc.). Hero tests are worse than useless, they are MISLEADING!
For example, consider if you’re going to buy a race car for driving on a race track, would you test drive it on ice? What would you learn from it? How relevant would it be in helping you assess how the race car will perform on the race track?
Similarly, hero tests create artificial test conditions that focus on artificial corner testing. The output is artificial results that don’t have a correlation with what the results would be on your production workloads (your objective).
What are the overall characteristics that should be tested with Flash?
In order to get an accurate assessment of how the AFA will perform with your production workloads, use ASAP to assess key areas (listed in the priority order).
Availability: How an AFA performs under failure and upgrades is not only critical, it’s tablestakes! Fail a controller under full load. Perform a SW upgrade. Fail (via pulling) dual SSDs within a RAID group concurrently. Observe data remains available. Measure 100% performance remains throughout.
Flash Scalability & Simplicity: Ensure you have the flexibility to purchase what you need now and grow online, as needed. Similarly, ensure you can free yourself and your organization from complexity. Test online capacity expansion and measure 100% performance remains throughout. Test no tuning at any layer (App, VM, FS, Array) and/or the addition of new layers (virtualization, appliances) is required. Test Support experience.
Flash Affordability: Production application workloads are reducible data sets. AFAs that deliver best-in-class data reduction (dedupe, compression) increase efficiency and lower $/GB usable. Measure the data reduction across mixed application workloads (not just VDI).
Performance: Nearly all AFAs can deliver high “hero benchmark” performance. Your challenge is to determine if the AFA can deliver consistent sub-ms average performance across mixed workloads (databases, virtual servers, virtual desktops, etc.). Test the performance with your production workloads and measure results.
In Summary…
You can watch the video here or download the IDC AFA Performance Testing Framework here . If you are interested in evaluating what a Pure Storage FlashArray can do for your business, we would love to hear from you .
