DirectFlash™ Fabric vs iSCSI with Epic Workload

With the launch of  DirectFlash™ Fabric Pure is demonstrate again its market-leading approach to  NVMe technology that was introduced well over three years ago with the FlashArray//M. This time,  NVMe Over Fabric RoCE (RDMA over Converged Ethernet) becomes the transport of choice for servers with high-speed Ethernet networks replacing old and CPU -bounded iSCSI protocol.

Epic is one of the main electronic health records (EHRs) in use at U.S. healthcare organizations today. With thousands of patients and healthcare providers simultaneously accessing and updating EHR information 24/7, on-site or via mobile apps, database response times determine how quickly data arrives in surgery or during patient visits.  Few industries are as storage-latency sensitive as hospitals and health systems but storage innovations from Pure have helped many healthcare organizations to deliver EHR data to providers much more quickly.

To demonstrate the advantages of DirectFlash Fabric from Pure for Epic EHRs, we use a tool developed by Epic called “Epic IO Simulator,” aka GenerateIO.

Performance test with Epic’s IO Simulator

Epic’s production OLTP database server is based on Caché from Intersystems — also known as the Operational Database, or ODB — and has well-defined storage response requirements for both reads and writes in order to ensure low response times for high performing interactive user workflows, to meet the needs of providers and patients in the health system.

The underlying InterSystems Caché database on which the Epic application runs has a write cycle of 80 seconds. To keep things humming, the flush of writes from the database must be completed within 45 seconds, well before the next write cycle begins, even under the heaviest of workloads.  And, in order to maintain appropriate response times and user experience, the ongoing voluminous random application reads must be reliably fast. The existing FlashArray, when configured properly, always meets and exceeds Epic’s requirements.

Epic uses its “Epic IO Simulator tool” (aka GenerateIO) to validate the performance of new storage configurations at customer sites; here we are using it to evaluate the performance of the storage array. Pure Storage FlashArray//X  with both iSCSI and NVMe oF shows great results with Epic’s IO Simulator, meeting the service-level agreement (SLA) requirements for write I/Os and exceeding the read latency expectation:

  • For Write IO : Longest write cycle must complete within 45 seconds
  • For Read IO: Random Read latencies (aka RandRead) must average below 2 ms

The “total IOPS” number is the number of read and write IOs (8K IO at 75/25 read/write ratio) generated by the tool before it reached the either of the read or write SLA limit .

Epic, for their production Database, requires the use of a hypervisor like VMware ESXi or Microsoft HyperV. Because the NVMe drivers were not fully qualified at the time of the test, we didn’t use any hypervisor.  The results would be slightly better bypassing the virtualization stack.

For this test, we compared a FlashArray//X90 using Purity 5.2 and both iSCSI Interface as well as the new DirectFlash Fabric. The servers we used for both tests were Intel-based servers,  running Red Hat Enterprise Linux RHEL 7.6.

Both servers had 2 ports using either NVMe of RoCE at 50Gb or iSCSI on ethernet at 25 Gb.

We found that the new DirectFlash Fabric offers a significant improvement, even when compared to the already exceptional performance of the iSCSI.

Results

DirectFlash Fabric vs iSCSI with Epic Workload
DirectFlash Fabric vs iSCSI with Epic Workload

When the Application “Epic IO Simulator” run, it generates several files, including a file called Summary.cvs which contains the summary of the test.

The following table aggregate the results found in the tests:

The 2 charts  “Epic GenerateIO longest Write Cycle” and “NVMe oF Epic GenerateIO RanRead” are calculated based on the summary.cvs from a run on iSCSI as well as a run on NVMe oF.

On the chart “Epic GenerateIO longest Write Cycle,” the red bar “45 Seconds Threshold” determines when the longest write cycle reach 45 Seconds. Then we use the number of Total IOPS as the maximum performance the FlashArray can reach. for the NVMe oF is reached at 216,794 IOPs while iSCSI had only 176,958 IOPs which represent a 18% improvement.

Ranread is Epic’s tool to measure latency at the server level, as seen in the chart “NVMe oF Epic GenerateIO RanRead” and the table NVMe oF has a ranread average of 0.42 milliseconds while iSCSI is only 0.69 milliseconds (for latency the lowest number is the highest performance) which represent 64% improvement.

Epic also measure the CPU utilization of the server, this is a very good indicator to demonstrate the fact that NVMe oF is a much “leaner” protocol than iSCSI. In this case when the application reach 45 seconds write flush, NVMe uses only 16% of the CPU while iSCSI uses 20% which is a 25% improvement.

 

Figure 1 : Epic GenerateIO longest Write Cycle
Figure 1 : Epic GenerateIO longest Write Cycle
Figure 2 : NVMe oF Epic GenerateIO RanRead
Figure 2 : NVMe oF Epic GenerateIO RanRead

 

 

 

 

 

 

 

 

 

 

 

 

Conclusion

This test results demonstrate that Pure’s DirectFlash Fabric is setting a new bar in performance for Epic workloads.  For most Epic customers seeking to migrate their workloads to new Ethernet environment at 25 Gb, 50 Gb or even 100 Gb this will provides a converged solution meet or exceed SLA requirements. Of course, Epic will need to validate this solution when the different virtualization technologies are fully supported (VMware, HyperV).

But the performance we have provided in this test proves that DirectFlash Fabric can sustain several times the largest requirements for Epic in term of IOPS and latency. New customers have even more options for configuration and performance with the DirectFlash Fabric when sourcing storage from Pure, allowing them to take advantage of the next step in technology with NVMe over Fabric.