Software Engineering Daily

Web3 promises to change the way we interact and transact online, but today’s dapps and wallets are hard to use. They are typically optimized for desktop machines and domain experts. In particular, it can be hard to bring value on-chain, and hard to know what to do once you’ve onboarded. Charlie Andrews-Jubelt is a software engineer who works on Valora, a mobile wallet, and Revo, a dapp that makes it easier to invest in DeFi and get compound interest. Charlie joins the show to talk about the engineering challenges of making web3 more accessible. Sponsorship inquiries: [email protected] Capital one: Visit https://capitalone.com/ML Keeper Security: Get started by visiting https://www.keepersecurity.com/ Fullstory : Go to https://fullstory.com/sedaily to activate your free one month trial. Skyflow: Visit https://skyflow.com Netapp: Check them out at https://spot.io/sedaily

While Kubernetes has many benefits, there is often a need for teams to deploy a monitoring and observability stack to troubleshoot issues that happen within the cluster and the applications themselves. ContainIQ, an out-of-the-box solution, allows engineers to monitor the health of their cluster and troubleshoot issues faster. ContainIQ is unique in its approach in that it was built with eBPF in mind, and is able to provide an APM like experience without being an APM. Matt Lenhard is the Co-founder and CTO of ContainIQ and joins the show to discuss the future of K8s monitoring and observability, as well as the unique technological approach he is taking with eBPF. Sponsorship inquiries: [email protected] New Relic: Sign up at https://newrelic.com/sedaily Netapp: Check them out at https://spot.io/sedaily Skyflow: Visit https://skyflow.com Capital one: Visit https://capitalone.com/ML Dragon Ball Legends : https://twitter.com/DB_Legends

Nearly all new tech companies build in a public cloud and established companies are rapidly migrating to the cloud from their on-prem data centers. But this move to the cloud can lead to a visibility problem. Cloud Providers offer not only Compute instances but also manage services like databases, blob storage, queues and more. It can be difficult for SRE teams and security departments to understand what is happening across a company's cloud accounts. Yevgeny Pats is the creator of CloudQuery, an open source cloud asset inventory powered by SQL. CloudQuery allows you to ingest and structure the resources in your cloud accounts so that you can query them using SQL. This allows SRE teams to understand the source of specific resources while security teams can ensure compliance with policies. In this episode we’ll discuss CloudQuery, Yevgeny’s entrepreneurial background and raising funding with an open source project Sponsorship inquiries: [email protected]. Capital one: Visit https://capitalone.com/ML Skyflow: Visit https://skyflow.com Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Netapp: Check them out at https://spot.io/sedaily Fullstory : Go to https://fullstory.com/sedaily to activate your free one month trial.

How to build a rocket ship while flying it? In this episode, Cos returns to the show to discuss the technologies and approach Brex chose to create an engineering organization and innovation engine for the long term, while navigating shifts as the company continued to scale at a rapid pace. Cos and Jeff also cover the nuts and bolts of programming languages and databases and strategies to break through performance bottlenecks. Cos is the CTO of Brex, whose mission is to reimagine financial systems so growing businesses can achieve their full potential. Sponsorship inquiries: [email protected]. SetApp : Head over to https://setapp.com/developers to get in touch. Fullstory : Go to https://fullstory.com/sedaily to activate your free one month trial. Capital one: Visit https://capitalone.com/ML Netapp: Check them out at https://spot.io/sedaily Cloud Bees: Visit https://www.cloudbees.com/

Convex makes storing your application's data as easy as using React state management. If you can use React hooks, you can also manage your backend data using Convex. James Cowling is a former Dropbox infrastructure engineer turned startup founder. James joins the show to discuss how Convex offers a simpler full-stack developer experience than the tools developers use today — without many of the compromises of other alternatives. This episode is guest hosted by Jackson Gabbard, a long-time Facebook product engineer who is now the CTO of Cord — the SDK for collaboration in any SaaS tool. Sponsorship inquiries: [email protected] Cox Automotive: Visit https://coxautotech.com/ to find career opportunities at Cox. Netapp: Check them out at https://spot.io/sedaily Datadog: Get started today with a free trial at https://softwareengineeringdaily.com/datadog Wix: Head over to https://Wix.com/Partners and reimagine what your agency can accomplish. Capital one: Visit https://capitalone.com/ML

Developers looking for read or write access to Ethereum, Polygon, IPFS or other Web3 networks in order to get their idea in the hands of users need reliable RPC endpoints they can count on, whether they are working on a hackathon project or running hybrid architecture for a production application. Infura provides a platform for developers to get access to those end points as well as tooling to help them build on Web3. With its origin being Ethereum, Infura has expanded to both EVM-compatible and non-EVM compatible blockchains, as well as decentralized storage. From wallets, DeFi, gaming, Metaverse, or NFT projects, Infura is relied upon by some of the most used and popular projects in Web3. With billions of daily requests handled, the team has built knowledge on how to scale with reliability. Co-Founder and General Manager of Infura, EG Galano joins the show to talk about Blockchain infrastructure and protocols. Sponsorship inquiries: [email protected]

Streaming data platforms like Kafka, Pulsar, and Kinesis are now common in mainstream enterprise architectures, providing low-latency real-time messaging for analytics and applications. However, stream processing - the act of filtering, transforming, or analyzing the data inside the messages - is still an exercise left to the receiving microservice or datastore, a custom programming exercise likely repeated over and over within an application. Stream processing tools such as Apache Flink and ksqlDB have been around for half a decade, but their complexity has hindered adoption. Decodable’s mission is to radically simplify processing on the stream with a SaaS platform based on Flink, and using only SQL which frees up developers to focus on what matters most. Eric Sammer is founder and CEO of Decodable and joins the show to discuss the potential of stream processing, its role in modern data platforms, and how it’s being used today. Sponsorship inquiries: [email protected] Fullstory : Go to https://fullstory.com/sedaily to activate your free one month trial. Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML Skyflow: Visit https://skyflow.com Cloud Bees: Visit https://www.cloudbees.com/

The Kubernetes ecosystem has drastically changed how development teams ship software. While Kubernetes has provided many advancements in cloud infrastructure, it has also left organizations with massive security blindspots. KSOC was created to give developers and security teams a single control plane to harden multi-cluster Kubernetes environments through event-driven analysis, least privilege enforcement, and remediation-as-code. Jimmy Mesta is the Co-Founder and CTO of KSOC and joins the show to discuss modern Kubernetes security challenges and how security teams need to prepare for a future where Kubernetes is the top attack target for adversaries. Sponsorship inquiries: [email protected] Skyflow: Visit https://skyflow.com Capital one: Visit https://capitalone.com/ML Datadog: Get started today with a free trial at https://softwareengineeringdaily.com/datadog Netapp: Check them out at https://spot.io/sedaily Cloud Bees: Visit https://www.cloudbees.com/

In software engineering, telemetry is the data that is collected about your applications. Unlike logging, which is used in the development of apps to pinpoint errors and code flows, telemetry data includes all operational data including logs, metrics, events, traces, usage, and other analytical data. Companies usually visualize this information to troubleshoot problems and understand patterns and opportunities in how their applications are used. The company New Relic is a modern observability platform built to optimize your entire software stack from one place. New Relic includes a Telemetry Data Platform that acts as a single source of truth for telemetry data. Built on top of that are tools for full stack observability to visualize and troubleshoot your data in milliseconds. When a problem does occur, New Relic’s applied intelligence will detect and understand the problem to help resolve it faster. Nic Benders is New Relic’s chief architect and GVP of engineering. As part of the engineering team since the early days of the company, Benders has been involved with everything from Agent development to ZooKeeper deployments and all the pieces and products in between. In his role as chief architect, Benders looks after the long-term technical strategy behind New Relic’s product and the experience of all the engineering teams who build it. He joins the show to discuss the engineering behind New Relic. Sponsorship inquiries: [email protected] Datadog: Get started today with a free trial at softwareengineeringdaily.com/datadog Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Capital one: Visit https://capitalone.com/ML Netapp: Check them out at https://spot.io/sedaily

One of the challenges with Microservices architecture is how you manage dependencies between your services when implementing workflows. Conductor is an open-source microservices and workflow orchestration platform. Boney Sekh co-founded Orkes Inc - a company focused on offering Conductor as a service. Boney joins the show to discuss how engineers leverage Conductor to build highly reliable distributed applications using. Sponsorship inquiries: [email protected] Get Stream: Check it out at https://getstream.io/ Netapp: Check them out at https://spot.io/sedaily Cox Automotive: Visit https://coxautotech.com/ to find career opportunities at Cox. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Capital one: Visit https://capitalone.com/ML

Notifications have typically been an area of a product where building in-house has been the only option. However, building a best-in-class notifications system that delivers a great customer experience requires a significant investment when you start to layer in complexities like batching, cross channel orchestration, and user preferences. Chris Bell of Knock joins the show to talk through how Knock can solve the challenges with building an in-house product notification system, and free up engineering time in the process. Sponsorship inquiries: [email protected] Get Stream: Check it out at https://getstream.io/ Netapp: Check them out at https://spot.io/sedaily Cox Automotive: Visit https://coxautotech.com/ to find career opportunities at Cox. Cloud Bees: Visit https://www.cloudbees.com/ Capital one: Visit https://capitalone.com/ML

The advent of the cloud introduced a new form of technical debt in which organizations can lose track of what infrastructure they have and how it relates to the business. While the cloud's native APIs offer some transparency into your infrastructure, these offerings are often described as necessary but not sufficient. When companies have a non-trivial question about their environments, getting answers can have a big cognitive overhead to get a clear answer. In this episode, I interview Tyson Kunovsky, Co-Founder and CEO at AutoCloud, creators of the open source project Cloud Graph. CloudGraph is a free and open source GraphQL API for AWS, Azure, GCP, and Kubernetes. CloudGraph provides a much more intuitive graphQL based interface to ask questions about your infrastructure in context of the relationships between components. Sponsorship inquiries: [email protected] Datadog: Get started today with a free trial at https://softwareengineeringdaily.com/datadog Capital one: Visit https://capitalone.com/ML RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Skyflow: Visit https://skyflow.com

Highlight is a tool that helps teams reproduce end-to-end user sessions to better understand their application. With Highlight, engineering teams can replay errors with high precision, which includes a complete session replay, outgoing network requests, dense stack traces and insight into the app's state management system (redux, apollo's cache, etc.). At the same time, product teams can analyze user activity, collaborate with engineering teams and find the reason for user drop offs to gain insight into UX. Jay Khatri is the CEO of Highlight and joins the show to talk about Browser Observability and what he and his team have built. Sponsorship inquiries: [email protected] Datadog: Get started today with a free trial at https://softwareengineeringdaily.com/datadog Mongo DB: Just visit https://mongodb.com/world-sedaily RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML

Lee Atchison spent seven years at Amazon working in retail, software distribution and Amazon Web Services. He then moved to New Relic, where he spent eight years scaling the company's internal architecture. Lee has deep expertise in building and managing fast growing web applications, and he used this knowledge to write architecting for scale from O'Reilly. Technical debt is a common problem for engineering teams. Technical debt is additional long-term work required to evolve or maintain an application due to a short-term decision in designing or building that application. Too much technical debt can reduce product innovation and lead to lower employee retention. In this episode, we'll discuss all things technical debt based on Lee’s recent article, Technical Debt Will Sink You. Sponsorship inquiries: [email protected]

There's an increasing regulatory and consumer pressure on companies to do a better job protecting sensitive customer data. Yet, despite this pressure, data breaches and compliance issues continue to plague the tech industry. Companies like Apple, Netflix, and Google have solved these challenges by pioneering a new type of technology, the data privacy vault. Skyflow is making this technology available to all companies through a simple API. Sean Falconer joins the show to discuss the features of a data privacy vault and the data security and compliance challenges it addresses. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Capital one: Visit https://capitalone.com/ML Skyflow: Visit https://skyflow.com. Netapp: Check them out at https://spot.io/sedaily Cloud Bees: Visit https://www.cloudbees.com/

Data-as-a-service is a company category type that is not as common as API-as-a-service, software-as-a-service, or platform-as-a-service. In order to vend data, a data-as-a-service provider needs to define how that data will be priced, stored, and delivered to users: streaming over an API or served via static files. Naqeeb Memon of Safegraph joins the show to talk through the mechanics of delivering data-as-a-service. Sponsorship inquiries: [email protected] Skyflow: Visit https://skyflow.com Cloud Bees: Visit https://www.cloudbees.com/ RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML

Blockchains were the first systems to allow guaranteed permanent storage of public data. As cryptocurrency technology has advanced, a rich ecosystem of permanent storage and compute has developed as well. One of these is Arweave, a system for permanent information storage. Sam Williams is a founder of Arweave and joins the show to talk through how Arweave works and his motivation for starting it. Sponsorship inquiries: [email protected] Mongo DB: Just visit https://mongodb.com/world-sedaily Cloud Bees: Visit https://www.cloudbees.com/ RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML

Data labeling allows machine learning algorithms to find patterns among the data. There are a variety of data labeling platforms that enable humans to apply labels to this data and ready it for algorithms. Heartex is a data labeling platform with an open source core. Michael Malyuk joins the show to talk through the platform and modern usage of data labeling systems. Sponsorship inquiries: [email protected] Get Stream: Check it out at https://getstream.io/ Netapp: Check them out at https://spot.io/sedaily Cox Automotive: Visit https://coxautotech.com/ to find career opportunities at Cox. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Capital one: Visit https://capitalone.com/ML

Real-time analytics are difficult to achieve because large amounts of data must be integrated into a data set as that data streams in. As the world moved from batch analytics powered by Hadoop into a norm of “real-time” analytics, a variety of open source systems emerged. One of these was Apache Pinot. StarTree is a company based on Apache Pinot that provides fast, real-time data analytics. Chinmay Soman joins the show to discuss Apache Pinot in relation to other real-time analytics platforms, and what StarTree has built on top of Pinot. Sponsorship inquiries: [email protected] Cox Automotive: Visit https://coxautotech.com/ to find career opportunities at Cox. Wix: Head over to https://Wix.com/Partners and reimagine what your agency can accomplish. Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML

Web3 is powerful but difficult to work with. Deploying blockchain nodes, accessing data, and performing staking operations are non-trivial engineering actions. To simplify web3, Ankr hosts APIs for node deployment, RPC, and staking. Josh Neuroth from Ankr joins the show to talk about modern web3 infrastructure. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Get Stream: Check it out at https://getstream.io/ Netapp: Check them out at https://spot.io/sedaily Skyflow: Visit https://skyflow.com today to learn how. Capital one: Visit https://capitalone.com/ML

The terminal is a necessary tool for any software engineer. In order to work quickly, developers have always customized their terminals to work for their specific application workflow. Fig is a reimagined terminal product that adds autocomplete and an entire app ecosystem to the existing terminal you are familiar with. In a previous show we talked through the basics of Fig. In today’s show we take a deeper dive into the functionality and engineering behind the product with the founders of Fig, Brendan and Matt. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily JS Nation: https://jsnation.com Skyflow: Visit https://skyflow.com Netapp: Check them out at https://spot.io/sedaily Capital one: Visit https://capitalone.com/ML

WP Engine is a domain specific cloud provider that hosts high performance WordPress infrastructure. This website, Software Engineering Daily, runs on WP Engine. Scaling a domain specific cloud provider for WordPress includes complexities at the level of the database, application, load balancer, and other areas. Brandon DuRutte from WP Engine joins the show to talk through his work. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Capital one: Visit https://capitalone.com/ML React Summit : https://reactsummit.com Netapp: Check them out at https://spot.io/sedaily Skyflow: Visit https://skyflow.com

Data loss can occur when large data sources such as Slack or Google Drive get leaked. In order to detect and avoid leaks, a data asset graph can be built to understand the risks of a company environment. Polymer is a data loss prevention product that helps companies avoid problematic data leaks. Yasir Ali is the founder of Polymer and joins the show to talk about the engineering and product vision of Polymer. https://www.polymerhq.io/ Sponsorship inquiries: [email protected] React Summit : https://reactsummit.com Skyflow: Visit https://skyflow.com Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Netapp: Check them out at https://spot.io/sedaily Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside.

The Presto/Trino project makes distributed querying easier across a variety of data sources. As the need for machine learning and other high volume data applications has increased, the need for support, tooling, and cloud infrastructure for Presto/Trino has increased with it. Justin Borgman is the CEO of Starburst, and joins the show to talk about the impact of Presto and how his company has architected its product. Sponsorship inquiries: [email protected] Cloud Bees: Visit softwareengineeringdaily.com/cloudbees to get a free trial. JS Nation: https://jsnation.com Skyflow: Visit https://skyflow.com Netapp: Check them out at https://spot.io/sedaily Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside.

Data integration infrastructure is not easy to build. Moving large amounts of data from one place to another has historically required developers to build ad hoc integration points to move data between SaaS services, data lakes, and data warehouses. Today, there are dedicated systems and services for moving these large batches of data. Airbyte builds open source data integration systems, and Michel Tricot from Airbyte joins the show to talk about the design and development of Airbyte. Sponsorship inquiries: [email protected] RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Capital one: Visit https://capitalone.com/ML Netapp: Check them out at https://spot.io/sedaily Get Stream: Check it out at https://getstream.io/ Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside.

Modern organizations eventually face data governance challenges. Keeping track of where data came from, what systems update it, in what ways updates can be made are just some of the issues to be tackled. Large organizations face additional challenges like training, onboarding, and capturing the institutional knowledge that leaves with the departure of key team members. As teams grow, these challenges often grow faster for unprepared organizations. Select Star helps companies unlock the full context of their data. Their solution automatically catalogs and documents your database tables and BI dashboards. In this episode I interview Shhinji Kim about the functionality of Select Star and how companies have achieved successful adoption. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Wix: Head over to https://Wix.com/Partners and reimagine what your agency can accomplish. Netapp: Check them out at https://spot.io/sedaily RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside.

The JavaScript supply chain includes numerous vulnerabilities due to its expansive nature and the long dependency chains. Socket is a new security company that can protect your most critical apps from supply chain attacks. They are taking an entirely new approach to one of the hardest problems in security in a stagnant part of the industry that has historically been obsessed with just reporting on known vulnerabilities. Feross is the Founder and CEO of Socket Security. He joins the show to talk about Socket’s approach to detecting and blocking supply chain attacks. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Netapp: Check them out at https://spot.io/sedaily Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside. Capital one: Visit https://capitalone.com/ML Monte Carlo: Visit https://softwareengineeringdaily.com/montecarlodata to learn more.

Loft is a platform for Kubernetes self-service and multi-tenancy. Loft allows you to control Kubernetes clusters with added multi-tenancy and self-service capabilities to get more value out of Kubernetes beyond simply cluster management. It allows for cost optimization, more efficient provisioning, and other features. Lukas Gentele joins the show to talk about Kubernetes multi-tenancy and the engineering behind Loft. Sponsorship inquiries: [email protected] Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Netapp: Check them out at https://spot.io/sedaily Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside. Capital one: Visit https://capitalone.com/ML Monte Carlo: Visit https://softwareengineeringdaily.com/montecarlodata to learn more.

Observability consists of metrics, logs, and traces. Lightstep is a company that builds distributed tracing infrastructure, which requires them to store and serve high volumes of trace data. There are numerous architectural challenges that come with managing this data. Ben Sigelman and Alex Kehlenbeck join the show to discuss the implementation of Lightstep. Sponsorship inquiries: [email protected] Work OS: To learn more and get started, go to https://softwareengineeringdaily.com/workos Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Strong DM: Start your free 14 day trial today at: https://strongdm.com/SEDaily Monte Carlo: Visit https://softwareengineeringdaily.com/montecarlodata to learn more.

The solution many turn to for capturing their streaming data is InfluxDB. In this episode, I interview Brian Gilmore, director of product management at Influx Data, about how real time applications achieve success built on top of InfluxDB. When most people hear the phrase Internet of Things, it typically evokes an image of connected devices we install in our homes. While this is a common use case, the true winner to date in IoT is probably industrial automation. While small improvements can yield big returns, and small errors can result in huge losses, it's critical to capture and elegantly handle telemetry data from industrial systems. Sponsorship inquiries: [email protected] Netapp: Check them out at https://spot.io/sedaily Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside. Get Stream: Check it out at https://getstream.io/ Capital one: Visit https://capitalone.com/ML Cox Communucations: Go to https://cox.career/techjobs to find out more about career opportunities at Cox Communications.

Terminals are a gateway to building and running software, but they have not been reimagined since their initial creation. Warp is a new kind of terminal with visual aids, autocomplete, documentation, customization, and other features. It includes GPU acceleration, its own UI framework, and other engineering systems to increase speed and performance. Zach Lloyd joins the show to talk through the creation of Warp and the future of developer tools. Sponsorship inquiries: [email protected] CODA: Head over to https://coda.io/sedaily Cox Communucations: Go to https://cox.career/techjobs to find out more about career opportunities at Cox Communications. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Strong DM: Start your free 14 day trial today at: https://strongdm.com/SEDaily Monte Carlo: Visit https://softwareengineeringdaily.com/montecarlodata to learn more.

Data infrastructure is a fast-moving sector of the software market. As the volume of data has increased, so too has the quality of tooling to support data management and data engineering. In today’s show, we have a guest from a data intensive company as well as a company that builds a popular data engineering product. James Densmore works at Hubspot, which produces tons of data, and Lior Gavish works at Monte Carlo Data, which sells a data quality product. Sponsorship inquiries: [email protected] Puppet:You can join Puppet for the conversation by registering at events.puppet.com/fireside. CODA: Head over to https://coda.io/sedaily Cox Communucations: Go to https://cox.career/techjobs to find out more about career opportunities at Cox Communications. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Work OS: To learn more and get started, go to https://softwareengineeringdaily.com/workos

Software engineering can be a surprisingly grueling career. It is both physically and mentally demanding to sit in front of a screen for hours on end producing code. During the pandemic, the strains of being an engineer became even more acute, as we were secluded in our homes, in some cases alone. Gabe Greenberg and Michelle Bakels work at G2i, a company that manages a large number of developers. G2i has a focus on developer health, which they believe is an underappreciated topic. They are also helping organize React Miami, a conference that takes place April 18-19, 2022. Sponsorship inquiries: [email protected] Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/fireside. Work OS: To learn more and get started, go to https://softwareengineeringdaily.com/workos RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Strong DM: Start your free 14 day trial today at: https://strongdm.com/SEDaily CODA: Head over to https://coda.io/sedaily

Crypto companies have cemented themselves as a company category that is not going away. Bitski is a company that makes web3 tooling and infrastructure, including a wallet and a collection of tools for creating and selling NFTs. Patrick Tescher joins the show to talk through the engineering behind crypto infrastructure and APIs. Sponsorship inquiries: [email protected] Doppler: Sign up via https://doppler.com/sedaily. Cox Communucations: Go to https://cox.career/techjobs to find out more about career opportunities at Cox Communications. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Strong DM: Start your free 14 day trial today at: https://strongdm.com/SEDaily CODA: Head over to https://coda.io/sedaily

Running a database company requires expertise in both technical and managerial skills. There are deeply technical engineering questions around query paths, scalability, and distributed systems. And there are complex managerial questions around developer productivity and task allocation. Sam Lambert is the CEO of PlanetScale, which is building modern relational database infrastructure. Before PlanetScale he spent several years on infrastructure at GitHub. He joins the show to talk about his work at PlanetScale and the vision for the company Sponsorship inquiries: [email protected] Puppet:You can join Puppet for the conversation by registering at https://events.puppet.com/event/fireside/ Cox Communucations: Go to https://cox.career/techjobs to find out more about career opportunities at Cox Communications. RudderStack: Software Engineering Daily listener at https://rudderstack.com/SED. Get Stream: Check it out at https://getstream.io/ New Relic: Sign up at https://newrelic.com/sedaily

GitOps is a deployment and infrastructure management strategy based around continuous delivery and Kubernetes, with Git at the center of deployment workflows. Policy management can be used to define permissions and rules around who can deploy and what constitutes safe deployment. There is a synergy between GitOps tooling and policy management tooling. Alexis Richardson is the co-founder and CEO of Weaveworks. Mohamed Ahmed founded Magalix, a security and compliance company that was acquired by Weaveworks. They join the show to talk about GitOps and Policy Management. Sponsorship inquiries: [email protected] Puppet:You can join Puppet for the conversation by registering at events.puppet.com/fireside. Influx Data: . To learn more and get started for free, visit https://influxdata.com/sedaily Strong DM: Start your free 14 day trial today at: http://strongdm.com/SEDaily RudderStack: Software Engineering Daily listener at http://rudderstack.com/SED. New Relic: Sign up at https://newrelic.com/sedaily

A company’s response to an incident defines how that company responds to an adverse, unexpected scenario. Kintaba automates the incident management process so teams can quickly report, respond, resolve, and reflect on major incidents collaboratively. Kintaba's incident response platforms gives you instant access to incident management processes based on the practices of companies like Facebook, Google, Stripe, and other successful and resilient organizations. John Egan is Cofounder & CEO at Kintaba and joins the show to talk about Incident Response. Attend The Incident Response Conference April 1, 2022 Sponsorship inquiries: [email protected] Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos Influx Data: . To learn more and get started for free, visit influxdata.com/sedaily Strong DM: Start your free 14 day trial today at: strongdm.com/SEDaily RudderStack: Software Engineering Daily listener at rudderstack.com/SED. New Relic: Sign up at newrelic.com/sedaily

SingleStore is a multi-use, multi-model database designed for transactional and analytic workloads, as well as search and other domain specific applications. SingleStore is the evolution of the database company MemSQL, which sought to bring fast, in-memory SQL database technology to market. Jordan Tigani is Chief Product Officer of SingleStore and joins the show to talk through the architecture and engineering of the SingleStore platform. Sponsorship inquiries: [email protected] Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos Influx Data: . To learn more and get started for free, visit influxdata.com/sedaily Cox Communucations: Go to cox.career/techjobs to find out more about career opportunities at Cox Communications. New Relic: Sign up at newrelic.com/sedaily

Compute resources continue to trend towards being cheaper, easier to use, and faster. Despite these positives, more compute demands more energy and therefore an increasing carbon footprint. With many companies committing to controlling their net carbon emissions, tools are required for engineers to not only measure their cloud infrastructure, but to make informed choices about how to set up the infrastructure. Steren Giannini is a senior product manager. He works on Google Cloud Carbon Footprint. In this interview, we discuss how Google Cloud collects and exposes this data to users and how those users are responding in their infrastructure approaches Sponsorship inquiries: [email protected] Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos Influx Data: . To learn more and get started for free, visit influxdata.com/sedaily Strong DM: Start your free 14 day trial today at: strongdm.com/SEDaily RudderStack: Software Engineering Daily listener at rudderstack.com/SED. New Relic: Sign up at newrelic.com/sedaily

Vantage is a system for optimizing cloud costs. It provides tools and interfaces for developers to analyze how they are spending on AWS resources, and has recently expanded into GCP as well. Vantage users gain an easy interface into their costs that would otherwise be hard to analyze via the raw AWS console. Ben Schaechter is the founder of Vantage. In a previous show he gave an overview of Vantage. In today’s episode he returns to discuss the engineering internals of Vantage and provide a macro perspective on where people are wasting money in the cloud. Sponsorship inquiries: [email protected] New Relic: Sign up at newrelic.com/sedaily Influx Data: . To learn more and get started for free, visit influxdata.com/sedaily Get Stream: Check it out at https://getstream.io/ Monte Carlo: Visit softwareengineeringdaily.com/montecarlodata to learn more. RudderStack: Software Engineering Daily listener at rudderstack.com/SED.

At Lyft, Ketan Umare worked on Flyte, an orchestration system for machine learning. Flyte provides reliability and APIs for machine learning workflows, and is used at companies outside of Lyft such as Spotify. Since leaving Lyft, Ketan founded Union.ai, a company focused on productionizing Flyte as a service. He joins the show to talk about the architecture and usage of Flyte, as well as how he is formulating the company around it. Sponsorship inquiries: [email protected] Cox Communucations: Go to cox.career/techjobs to find out more about career opportunities at Cox Communications. Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos Get Stream: Check it out at https://getstream.io/ Strong DM: Start your free 14 day trial today at: strongdm.com/SEDaily RudderStack: Software Engineering Daily listener at rudderstack.com/SED.

DuckDB is a relational database management system with no external dependencies, with a simple system for deployment and integration into build processes. It enables complex queries in SQL with a large function library, and provides transactional guarantees through multi-version concurrency control. Hannes Mühleisen works on DuckDB and joins the show to talk about query engines and OLAP system Sponsorship inquiries: [email protected] Rudder Stack:Visit datastackshow.com/live to register today. Influx Data: To learn more and get started for free, visit influxdata.com/sedaily Get Stream: Check it out at https://getstream.io/ Monte Carlo: Visit softwareengineeringdaily.com/montecarlodata to learn more. Cox Automotive: Visit COXAUTOTECH.COM to find career opportunities at Cox.

Substack is a platform for subscription-based content, covering newsletters, podcasts, and soon video. The design and speed of the platform are notable–with a minimalistic format that simply gets creators paid for its work, Substack has gained massive popularity. Chris Best is the CEO of Substack and joins the show to talk about the platform, architecture, and engineering of Substack. Sponsorship inquiries: [email protected] Private Internet Access: .Go to https://privateinternetaccess.com/SEDaily InfluxData: To learn more and get started for free, visit influxdata.com/sedaily GetStream:Check it out at https://getstream.io/ MonteCarlo: Visit softwareengineeringdaily.com/montecarlodata to learn more. CoxAutomotive: visit COXAUTOTECH.COM to find career opportunities at Cox.

Customer data pipelines power the backend of many successful web platforms. In a customer data pipeline, data is collected from sources such as mobile apps and cloud SaaS tools, transformed and munged using data engineering, stored in data warehouses, and piped to analytics, advertising platforms, and data infrastructure. RudderStack is an open source customer data pipeline system that pulls together this disparate functionality. In a previous episode, we covered the basics of RudderStack. In today’s show, we dive deeper into the engineering of RudderStack with returning guest CEO Soumyadeb Mitra. Sponsorship inquiries: [email protected] InfluxData:To learn more and get started for free, visit influxdata.com/sedaily StrongDM: Start your free 14 day trial today at: strongdm.com/SEDaily GetStream: Check it out at https://getstream.io/ Monte Carlo: Visit softwareengineeringdaily.com/montecarlodata to learn more. Cox Automotive: visit COXAUTOTECH.COM to find career opportunities at Cox.

Managing Kubernetes nodes leads to operational complexity, security issues, and nodes that are perhaps more expensive to run than necessary. Deferring the node management to an underlying platform abstracts away these problems and can improve operations. Madhuri Yechuri runs Elotl, a nodeless Kubernetes platform. She joins the show to talk about the architecture and purpose of her company. Sponsorship inquiries: [email protected] Rudder Stack: Sign up today... or just get a free t-shirt for being a Software Engineering Daily listener at rudderstack.com/SED. Influxdata: To learn more and get started for free, visit influxdata.com/sedaily Monte Carlo: Visit softwareengineeringdaily.com/montecarlodata to learn more. Getstream: Check it out at https://getstream.io/ Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos

Retool is a company that allows customers to build complex internal tools using a high level GUI. Users configure the relationships between these different tools, giving them the ability to build applications even without much background in engineering. Of course, having some engineering expertise helps, as users can write JavaScript to interact with the higher level components. Snir Kodesh is the head of engineering at Retool, and he previously worked at Lyft as a director of engineering. He joins the show to talk about his work at Retool, how it compares to Lyft engineering, and the lower level challenges of building the platform. Sponsorship inquiries: [email protected] Get Stream: Check it out at https://getstream.io/ Cox Automative: Visit COXAUTOTECH.COM to find career opportunities at Cox. Influx Data: To learn more and get started for free, visit influxdata.com/sedaily Private Internet Access: .Go to https://privateinternetaccess.com/SEDaily Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos

The software supply chain consists of packages, imports, dependencies, containers, and APIs. These different components each have unique security risks. To ensure the security of their software supply chain, many developers use tools to analyze and scan their infrastructure for vulnerabilities. Barak Schoster works at Bridgecrew, a DevSecOps cloud security platform. He joins the show to talk about the risks of the modern software supply chain and what his company does to alleviate it. Sponsorship inquiries: [email protected] Get Stream: Check it out at https://getstream.io/ Private Internet Access: .Go to https://privateinternetaccess.com/SEDaily Work OS: To learn more and get started, go to softwareengineeringdaily.com/workos Influx Data: To learn more and get started for free, visit influxdata.com/sedaily Cox Automotive: visit COXAUTOTECH.COM

The data lake architecture has become broadly adopted in a relatively short period of time. In a nutshell, that means data in it’s raw format stored in cloud object storage. Modern software and data engineers have no shortage of options for accessing their data lake, but that list shrinks quickly if you care about features like transactions. Apache Hudi is a platform for building streaming data lakes that is optimized for lake engines and batch processing. In this episode, I interview Vinoth Chandar, creator of the Hudi Project and Founder and CEO at Onehouse. Sponsorship inquiries: [email protected] Support for the show comes from StackOverflow stackoverflow.com/teams/sedaily Coda coda.io/sedaily Private Internet Access privateinternetaccess.com/SEDaily RudderStack datastackshow.com/live

National Instruments develops software and hardware for engineering in a wide variety of domains, from aerospace to government technology to application testing. The interface between hardware and software presents a variety of difficult engineering challenges. Luke Schreier is a Senior Vice President at National Instruments and joins the show to discuss the engineering and management of the company. Sponsorship inquiries: [email protected] Support for the show comes from StackOverflow stackoverflow.com/teams/sedaily Coda coda.io/sedaily Private Internet Access privateinternetaccess.com/SEDaily Honeycomb softwareengineeringdaily.com/honeycomb RudderStack datastackshow.com/live

Automating video search requires a data pipeline that extracts metadata from videos and allows users to annotate the video with information that correlates with that metadata. The video needs to be segmented into intervals that define the search space, and the search space needs to be queryable by a user. Sieve Data is an automated video search platform that enables developers to add video search and analytics to internal tools and applications. Mokshith Voodarla is the CEO of Sieve Data and joins the show to talk about video search and what he and his team have built. Sponsorship inquiries: [email protected] GetStream: Check it out at https://getstream.io/ CapitalOne: Visit capitalone.com/ML Mparticle: Visit mparticle.com MONTE CARLO: Visit softwareengineeringdaily.com/montecarlodata to learn more. TotallyBio: To learn more go to TotallyBio.io.