Pipeline Data Engineering Academy home blog pages letters

The Data Janitor Letters - October 2021

Data engineering salon. News and interesting reads about the world of data.

Eating the Cloud from Outside In
Shawn Wang, Developer Experience, Temporal.io

AWS is playing Chess. Cloudflare is playing Go.


Why Lightspeed invested in ClickHouse: a database built for speed
Gaurav Gupta, VC, Lightspeed Venture Partners

$250M Series B financing of ClickHouse.


Day In The Life Of A Data Engineer — What Do Data Engineers Do?
SeattleDataGuy

There’s no better time to jump into the world of data engineering.


ROAPI: An API Server for Static Datasets
Mark Litwintschik, #bigdata Consultant

ROAPI is an API Server that exposes CSV, JSON and Parquet files without the need to write any code.


Announcing Streamlit 1.0! 🎈
Adrien Treuille, Co-Founder and CEO, Streamlit

Streamlit used to be the simplest way to write data apps. Now it's the most powerful.


Function pipelines
David Kohn, Developer, Timescale

Building functional programming into PostgreSQL using custom operators.


Implement a slowly changing dimension in Amazon Redshift
Milind Oke and Bhanu Pittampally, Amazon


DBT at HomeToGo. Creating a scalable framework
Gijs de Kruif, HomeToGo

Nothing new under the sun.


How to Execute Pandas Workloads in a Distributed Manner With Apache Spark
Hyukjin Kwon and Xinrong Meng, Databricks

If you have to do it you have to do it :facepalm:


Bash functions are better than I thought
Gary Verhaegen, Senior Software Engineer, Digital Asset

Given all that, I simply do not understand why people keep recommending the {} syntax at all.