Data engineering salon. News and interesting reads about the world of data.
Or why in five years every organization will have an Analytics Engineering team.
It’s important to note that the anti-patterns we’ll discuss below are specific to startups.
The current discourse on data can get a little tiring because of its over focus on tooling.
With some tweaking Postgres can be a great data warehouse. Here's how to configure it.
My larger point here is when you have a hammer (spark) everything looks like nail.
I personally believe that Airflow + Docker it’s a good combination for flexible, scalable, and hassle-free environments for ELT/ETL tasks.
When designing systems, less is more.
All the development was done on the trusty Sun Ultra 10 I had taken out a $10,000 loan to purchase when starting up the company.
SQL is king. Airflow is shit, yes.