For a number of years, market analysts have been writing about the evolution of traditional data warehouses. From scalable distributed file system for large distributed data-intensive applications (e.g., Google File System) to new programming frameworks to support the processing and storage of extremely large data sets in a distributed computing environment (Hadoop, Spark, et cetera), data warehouses have evolved.
And so has their underlying technology.
Data Warehouses and Data Lakes
Market analysts agree the data warehouse, built upon a relational database, will continue to be the primary analytic database for storing much of a company’s core transactional data:
- sales transactions
- customer data
- financial records
These data warehouses will be augmented by big-data systems, or data lakes. Data lakes are repositories for large volumes of machine-generated, unstructured data:
- log files
- social media data
- videos and images
The data lake will be also used as a repository for more granular transactional data or older transactional data which is not stored in the relational data warehouse.
Even though this new information architecture consists of multiple physical data repositories and formats, the logical architecture is a single integrated data platform, spanning the relational data warehouse and the data lake.
How much is all this data worth?
Data is the new business currency, but it’s only valuable if it’s accessible.
Data’s value comes from the ability to perform real-time, ad-hoc analytics with it, correlating it to various internal and external sources to create a bridge between your data warehouse and data lake stores.
Organizations continue to generate and store large volumes of digital data, and they need solutions that are ready for that growth. The volume, velocity and variety of data makes it difficult to ingest, store and optimally respond to both traditional transactional reporting and newer Big Data workloads. The data growth in source systems has an impact in loading and query performance. The environment hosting it must be flexible and scalable.
FlashBlade® Is Up to the Task
New architectures and technologies are changing the landscape of the modern data warehouse. More powerful computing and all-flash storage platforms are available for increasingly lower costs. At the same time, relational database management systems (e.g., Oracle 12c, PostgreSQL and others) continue to evolve and provide new features to address some of the challenges mentioned above.
Modern hardware data platforms using all-flash arrays empower businesses to implement agile ways to obtain the information needed. It also enables the analytical capabilities that are a critical part of an organization’s digital transformation and competitive strategy.
Learn how to diagram a data warehouse, and lean more about FlashBlade//S, the last scale-out solution you’ll ever need.
