Software development has evolved from a traditional waterfall model to agile and parallel development across industries. In high tech, companies now optimize and automate chip-design workflows to use development and design tools from blueprints to tapeout leading into manufacturing.
Managing multiple types and versions of source code during the software development life cycle (SDLC) is a key requirement for many organizations that use a version-control system like Perforce Helix Core. Source code includes proprietary computer programs written by programmers and developers to accomplish certain technical objectives. To function, source code requires additional development tools to compile and convert it into binaries or executables.
Modern application development processes are automated. They operate iteratively to identify and prioritize bugs in early development phases. The transition to “shift left” has led to increased operational concurrency and speed to complete development cycles with improved code quality. At the same time, it has resulted in massive data growth and increased infrastructure cost.
Using Pure Storage as a data platform has advantages compared to local storage for testing code changes and integrating the changes to the main code branch. A hybrid architecture with Pure Storage FlashArrayTM and FlashBladeTM products for Helix Core allows you to scale performance and capacity with optimized data efficiency.
Figure 1 shows the components of Perforce Helix as configured on Pure FlashArray and FlashBlade products. For more information access to the white paper titled: Digital Asset Management with Perforce Helix Core on Pure Storage.
We ran a comparison to identify the benefits of using Pure products as the data platform.
FlashArray low-latency block storage improves performance of latency-sensitive applications like databases. We configured the Helix Core database along with its journals, checkpoints, and replicas/edge servers on a FlashArray over iSCSI for high performance.
FlashBlade performs like a true data hub. It’s purpose-built to handle all the various workloads generated during product development, including software development, chip design, artificial intelligence (AI), and analytics. The advanced scale-out architecture supports network file services (NFS) and object storage on the same platform. We configured the Helix Core depot and server logs on the FlashBlade for linear performance scalability and optimized data growth.
A few things really stood out during the process of validating Helix Core on this hybrid architecture compared to local SSD-based storage on the servers without Pure elements. We used some commonly used Helix file-system operations including sync, fstat, edits, and submits for this validation. The Pure Storage hybrid architecture provided the following improvements:
- There was a 14% improvement for p4 sync operations while creating user workspaces on FlashBlade over NFS compared to local SSD. Instead of a single commit server, our setup used three edge servers to provide load distribution and improve both network performance and handling of metadata operations.
- Multiple iterations of test results showed about 11% improvement in the job completion time for Helix Core operations including sync, submit, filelog, and fstat in the user workspaces provisioned on the FlashBlade. This variation is primarily due to job concurrency. The FlashBlade excels at performance under high concurrency compared to local SSD storage at scale.
- Provisioning user workspaces on FlashBlade over NFS provides better capacity and performance scalability along with manageability.
- FlashBlade’s compression feature demonstrated up to 1.4:1 space savings during the test. Storage efficiency and scalable performance are among the most important requirements for better ROI.
- Network utilization is one of the key components to watch out for. If the network is the bottleneck, then increasing the parallel threads at the Perforce Helix layer or the number of Helix clients does not matter. In reality, network saturation added delay to the job completion time.
It’s also very important to understand about Helix file compression, particularly text files that directly or indirectly impact the network bandwidth. It is recommended that you enable file compression. Although compression and decompression require CPU overhead, it does improve network bandwidth from the clients to the Helix server. Uncompressed files have minimum CPU utilization on the servers but don’t optimize network bandwidth. FlashBlade has compression enabled at the array level that helps with uncompressed files managed by Helix Core.
While small organizations may use servers with local SSD for Perforce Helix Core, we recommend that mid- to large-sized enterprises utilize shared storage with DirectFlash to manage the digital assets. A hybrid architecture on Pure FlashArray and FlashBlade accelerate various Helix database and file-system operations with linear capacity and performance scalability.
The performance results discussed in this blog highlights the benefits of flash for Perforce Helix Core. In the second part of the blog, I will highlight how to rapidly reseed the Helix database from any disaster.
