There is a bit of a confusion between Data Warehouse vs Data Lake or ETL vs ELT. I hear that Data Warehouses are not used anymore, that they are replaced by Data Lakes altogether, but is that true? And why do we need Data Warehouses anyway? I will go into that as well as the definitions of both pluses explain the differences between them.
Data Warehouse vs Data Lake
Data Warehouse definition
A Data Warehouse, in short DWH and also known as an Enterprise Data Warehouse (EDW), is the traditional way of collecting data as we do since 31 years. The DWH serves the purpose of being the data integration from many different sources, the single point of truth and the data management meaning cleaning, historize and data joined together. It provides greater executive insight into corporate performance with management Dashboards, Reports or Ad-Hoc Analyses.
Various types of business data are analysed with Data Warehouses. The need for it often…
The 5 top most searched Data Warehouse Automation tools on the market compared with GoogleTrends is telling you that WhereScape is first before TimeXtender and BiReady (new Attunity Compose) over the last year. See the picture in full size or go directly to GoogleTrend comparison and change to your own needs.
Although the analysis is not representative, it still gives some insights and a good overview to size and presumably usage compared to each other, worldwide. Please consider that WhereScape and TimeXtender have more search results as the company name is the same as their product, meaning some of them are dedicated to the company name rather the Data Warehouse Automation (DWA) tool itself. And BimlFlex just published their first release and biGENiUS is rather new to market their product actively, they will probably increase slightly in the soon future.
Data Warehouse Automation Tools on the market
As you can imagine, there are plenty of…
This article is for you if you considering to use Data Warehouse Automation (DWA) and asking yourself why you should use Data Warehouse Automation tools what does it do for you. After I explained in my previous blog Why Data Warehouse Automation is not more popular, you will find the why and what of Data Warehouse Automation in this second post of the series.
Why automate your Data Warehouse?
Every industry has used automation to increase productivity, reduce manual effort, improve quality and consistency, and speed delivery. Henry Ford introduced the assembly to produce automobiles, and today Uber and countless other startups use the Internet and digital processing to reduce friction in commercial transactions. Thus, the time has come to introduce automation to data warehousing.
Pointed out by Eckerson Group.
I would say it like this. In a society where time flys remarkably fast and data became the new gold, it’s crucial to have proper analyses…
I was working with a Data Warehouse Automation (DWA) tool for a little more than a year, and I have to say I loved it. As a BI developer you could focus on the challenges you had in dimensional modelling, what granularity should you have the fact tables and going crazy with the business requirements and everything fast, consistent and tested!
But why is Data Warehouse Automation not used more often and more popular? I’m asking that myself more and more. That’s why I’m writing a series of blog posts all about DWA. In this first blog, I’m trying to find possible reasons behind and also argue for DWA, and why we should use it more often.
Every one needs to make data driven decision faster, why not use a generator which gives you answers in days instead of months..?
What do I mean by that? Many people and therefore many companies fear…
Very long time ago, 1986, 31 years ago to be precise, IBM in Europe created the very first architecture of a data warehouse. And it seems to be a masterpiece as it didn’t change much since. Though how can we improve or bring some innovation into Data Warehouse business in times everyone is talking about big data, data lakes, Internet of Things (IoT), predictive analytics, Data Vault, etc.?
No matter how we want to improve the architecture, it has to be automated as much as possible. Nowadays it became too slow for serving the business needs doing it the traditional way. However, I don’t think DWH will go away anytime soon (see more DWH vs Data Lake). I strongly believe that DWA tools are the future and will boost the Data Warehousing reputation back to earlier years.
That’s why I wrote this blog series all about DWA. You can start with the first blog…