In October, Pure Storage® announced a partnership with Confluent to offer the first on-premises tiered storage solution for streaming data. The solution leverages the Tiered Storage feature in Confluent Platform 6.0, which is part of Project Metamorphosis 6.
Every month, Confluent releases new products and capabilities based on a theme. This month, as part of Project Metamorphosis Month 7, Confluent is focusing on reliability, security, and proactive support—concepts that are deeply embedded in our DNA at Pure. In this blog post, we’ll take a look at how we’re supporting this month’s themes.
Read the whitepaper: “The Storage Reliability Imperative: Evolving Requirements in the Era of Digital Transformation”
Streaming Data 101: Reliability
Confluent introduced multiple reliability features, including:
-
Latest Stable Kafka: Provides upgrades with zero downtime to ensure you’re always on the latest stable version of Kafka
-
Operator: Automates rolling upgrades
At Pure, we refer to this process as a non-disruptive upgrade (NDU). An NDU is a software upgrade, hardware expansion, and/or replacement that doesn’t impact data availability or performance. Software upgrades, capacity expansion, and controller upgrades (including next-gen controller upgrades) take place non-disruptively. NDUs are a hallmark of the Pure Evergreen™ Storage program, which provides you with the latest technology and features through completely non-disruptive upgrades while preserving your prior investment in Pure solutions. And it’s all included in the cost of regular maintenance.
A great example of a Pure customer taking advantage of the Evergreen program is Liquidnet Holdings, a financial tech firm based in New York City. Liquidnet’s application development and analytics teams are deploying Elasticsearch, Spark, and Kafka as part of a consolidated data hub on FlashBlade®.
“With Evergreen, you can keep the same footprint and still add capacity and benefit from new technology,” says Mani Venkateswaran, global head of product support. “Pure Storage is more efficient in a data-center environment, lowering operating costs and eliminating forklift upgrades.”
Security
Confluent announced a new “Bring Your Own Key” feature, which allows you to manage your own encryption keys for at-rest data on dedicated clusters. This builds on the existing encryption feature that enables you to encrypt data at rest or in transit between Kafka and clients.
For on-premises deployments, FlashBlade complements Confluent Platform, an enterprise event streaming platform, with encryption that’s built-in, always-on, and always in-line. It’s included with your subscription at no additional charge. There’s no impact on performance, no administrative overhead, no key management, and no additional upgrades needed. Pure accomplishes this by securing data with AES-256-bit encryption without impacting performance and while maintaining full data-reduction capabilities. Pure encryption is:
- Federal Information Processing Standards (FIPS) 140-2 certified
- National Institute of Standards and Technology (NIST) compliant
- National Information Assurance Partnership (NIAP)/Common Criteria validated
- Payment Card Industry Data Security Standard (PCIDSS) compliant
One of many companies seeing the benefits of having Pure encryption is Reval, the leading global provider of scalable cloud solutions for treasury and risk management. According to Jogen Mehta, director of IT hosting, “In the old system, you could turn on encryption, but it resulted in a significant hit on performance. With Pure, you get encryption at rest without giving up any performance.”
Proactive Support
Confluent’s release of Proactive Support delivers continuous, real-time analysis of metadata to alert you of potential problems before they arise.
Pure1® is a perfect complement to Confluent Proactive Support. Engineered to be self-driving, this AI-driven infrastructure management tool helps you proactively identify potential issues and solve problems before they occur, saving you time, hassle, and precious resources.
By analyzing historical data, Pure1 constantly scans for known issues in an array and generates support tickets to notify you so you can resolve potential issues before they become problems. The self-driving nature of Pure1 also gives you insight into performance and capacity growth, helps predict upgrade improvements, and optimizes resource utilization.
NTT Data is just one of the organizations that has benefited from Pure1. Since 1988, NTT DATA has played a significant role in establishing and advancing IT infrastructure. Headquartered in Tokyo, with business operations in more than 50 countries and regions, NTT DATA provides a variety of professional services such as consulting, system development, business IT outsourcing, and more. NTT DATA uses Pure1 to plan for performance and capacity growth and solve problems before they happen.
“NTT DATA has benefited tremendously from the Pure1 support platform,” said Yuji Chigita, senior manager, digital business solution division. “Pure1 is able to proactively identify issues based on our past activity, alert our team if a problem is detected, and resolve it before it occurs. Self-driving storage puts us at ease knowing that any issues will be proactively resolved. Pure1’s AI-driven management also allows us to run simulations so that we can gauge when, and if, our arrays will become low on capacity.”
Global enterprises are rapidly undergoing business transformation by leveraging Kafka for the move from batch to real time, and often these organizations are under regulatory pressure, which complicates the transition. Together, Confluent and Pure Storage are delivering the reliability and security features organizations need to confidently deploy Kafka at scale.
Accelerating Software Development with Streaming Data Analytics
