60: Architecting a Boring Stream Processing Tool With Ashley Jeffs of Benthos
This week on The Data Stack Show, Eric and Kostas talk with Ashley Jeffs, the creator and maintainer of an open source project for data stream processing called Benthos. Ashley describes the journey of Benthos from starting out as just a personal “weekend warrior” project of his to becoming a full-fledged project that he’s maintained for five years.
Audio is streamed directly from the publisher (afp-928695-injected.calisto.simplecastaudio.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
Highlights from this week’s conversation include:
- A brief overview of Ashley’s background (2:47)
- Benthos’ creation and the problems it was meant to address (4:01)
- Use cases for Benthos (18:25)
- Key features of Benthos that make it stand out (22:23)
- Adding windowing to Benthos for fun (29:23)
- The highs and lows of maintaining an open source project for five years (32:17)
- The architecture of Benthos (36:23)
- The importance of ordering in streaming processing (42:15)
- Gaining traction with an open source project (53:21)
- Benthos’ blobfish mascot (58:03)
The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.
RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.