Deliver Better Software with Continuous Testing

In the rush to release new products, quality can suffer if you take shortcuts on testing. See how continuous testing and the right infrastructure can help.

Deliver Better Software with Continuous Testing

For years, companies in every industry have been pivoting to an online, digital-first approach for engaging customers. Even the most die-hard “physical media” companies like video game manufacturers are designing their products to be digital-only and using continuous testing to accelerate time to market.

The transition to digital-first makes sense. It enables faster interactions, reduces theft, and creates tremendous opportunity for ongoing enhancements to customer experience, either through improvement to an app or delivery of new content. 

But it also creates tremendous risk.

Speed – Control = Dangerous

Especially in a Digital-first World

As developers race to release new offerings in less time — either to beat competitors to market or serve up high-demand features to existing customers — they may cut short stress testing and edge case analysis. Or run tests after they’ve released code.

This practice opens even the best companies to financial risk and brand damage. For instance, the Sony PlayStation 5 – which now has a digital-only model – is already having issues with download queues, and Cyberpunk 2077, one of the marquee games for the platform, was pulled from the Playstation online store because of quality issues.

Customers will not tolerate this, on any medium. As a proof point, recently reviewed a report by Testlio on the state of commercial and retail app testing. The report indicates that “50% of users will not download an application with a 3-star rating, and 85% won’t download one with two stars.” 

Net, cutting software testing is a dangerous idea.

For this reason, many organizations are investing in log analytics pipelines to enable continuous, high-speed testing of code. These pipelines automate and accelerate functional, stress, and ecosystem testing, but to get testing done on time, every time, organizations need an infrastructure that can:

  • Scale performance to enrich/tag data and run thousands of tests every day
  • Store TBs of logs, metadata, and historical data for trend analysis 
  • Support containerized workloads for on-demand changes to the test environment

Insights on Delivering Pipelines that Scale

How Pure Storage Handles the Challenge

At Pure Storage®, we recognize the opportunity and risk that come with a digital-first approach to customer experience. 

We believe customers should have access to continuous innovation with regular updates to the Purity Operating Environment. That might be for patches or — unlike other vendors who gouge their customers — providing new software functionality without an upcharge.

But ensuring updates are high quality and delivered in a timely fashion isn’t simple. We quickly realized that traditional testing models were insufficient, especially as the number of concurrent tests grew from hundreds to hundreds of thousands of tests per day. 

To succeed, we adopted a CI/CD development process with continuous testing (Figure 1). And In this new solution brief on CI/CD development with continuous testing, we’re excited to provide an inside look at how we use FlashBlade® to deliver an analytics pipeline with superior scale and faster time to issue resolution.

CI/CD development process with continuous testing
Figure 1: High-level example of the CI/CD development process at Pure.

We hope this gives you some unique ideas on how to enhance your own pipelines and demonstrates why you can be confident that Pure Storage is the best platform for data analytics. 

Because at Pure, we aren’t just the vendor. We’re the customer.