Data Warehouse 3.0: It’s Gone Virtual!
A Virtual Data Warehouse is next. In this model we continue to take advantage of inexpensive and flexible cloud storage (S3, Azure Storage, ect) but no longer do we need to adhere to the ELT model of carefully building out curated data models in the hope they meet the needs of analysts. Virtual Data Warehousing lets anyone curate on-demand “datamarts” directly from cloud storage or any other data source and connect this to their favorite analytics tool easily. Complex semi-structured data, relational tables, you name it — whatever they have is easily accessed and made ready for use. In the virtual model, you don’t need ETL or ELT — data never moves from the initial cloud storage. In the virtual model users create virtual “views” of the data that are specifically suited for analytics. In essence, you have a virtual data mart that can be shared, published, and modified (given correct permissions) by anyone to be exactly what they need. Nothing more.
The SlamData Virtual Data Warehouse architecture
What’s the advantage of this approach? There are many, but first and foremost it’s about empowering end users and analysts so they don’t need to rely on data integration engineers or rigid ETL/ELT models. No more relying on curated predefined data models.
Second, it’s about agility. With this model companies can easily combine lots of data into cloud stores like S3 and make it immediately available for use by end users and analysts, bypassing the complex ETL/ELT process, no extensive data relocation, and no tedious data prep.
It’s Much Cheaper
Finally, it’s about lowering costs. Leveraging inexpensive cloud storage, cutting out expensive middlemen and creating curated data models lowers costs. And virtual data warehouses move less data, so data and transport costs are lowered.