Data engineering salon. News and interesting reads about the world of data.
It was a wild year for the database industry, with newcomers overtaking the old guard, vendors fighting over benchmark numbers, and eye-popping funding rounds. We also had to say goodbye to some of our database friends through acquisitions, bankruptcies, or retractions.
A detailed guide to help you navigate the modern data stack and build your own platform using open-source technologies.
I think the experience shows that PoCs using “production infrastructure” can expose pitfalls that might appear in a real implementation.
As long as you don't expect to need tens of thousands of small writes per second, thousands of large writes, or long-lived write transactions, it's highly likely that SQLite will support your usecase.
While onboarding customer after customer this year I've noted a few key things everyone should put in place right away - to either improve the health of your database or to save yourself from a bad day.
I don’t want to minimize deep learning’s achievements in reducing the toil involved in building feature pipelines, I’m still constantly amazed at how effective they are. I would like to see more emphasis put on feature engineering in research and teaching though.
I’m gonna hit you with hard truth my friend.
Why I don't write useful software unless you pay me.