Kafka Streams with Jay Kreps
Software Engineering Daily · softwareengineeringdaily.com
Audio is streamed directly from the publisher (traffic.megaphone.fm) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
Kafka Streams is a library for building streaming applications that transform input Kafka topics into output Kafka topics. In a time when there are numerous streaming frameworks already out there, why do we need yet another? To quote today’s guest Jay Kreps “the gap we see Kafka Streams filling is less the analytics-focused domain these frameworks focus on and more building core applications and microservices that process data streams.”
Jay is the CEO of Confluent, a company that is building Kafka technology, and he is one of the original authors of Kafka. We began with a brief history of why Kafka evolved to be the message broker of choice for so many data engineering stacks, then we got into a discussion of streaming–what is streaming? How does Kafka communicate with streaming frameworks? Our conversation concluded with a discussion of Kafka Streams and the future of the Kafka data platform.