Data Pipelines: Transforming Raw Data into Actionable Insights
Audio is streamed directly from the publisher (media.transistor.fm) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
This podcast episode talks about data pipelines, which are used to move data from one place to another and transform it into a more usable form. The podcast compares data pipelines to water pipelines, where raw data is like dirty water that needs to be cleaned and enriched.
The podcast covers topics such as batch and real-time pipelines, serverless computing, ETL, and the challenges of building and managing data pipelines. Michael and Chris also discuss the importance of data quality, involving the right people, and understanding business objectives. They emphasize the need to view data pipelines as a product that requires ongoing maintenance and support, and they provide tips for managing data pipelines in organizations.
Check out Data Pipelines explained here in this video: https://youtu.be/6kEGUCrBEU0