Software Engineering Daily

The complexity of building web applications seems to have grown exponentially in the last several years. This added complexity may bring power, but it can also make applications brittle, costly, and difficult to maintain. Suborbital is an open-source project with a goal of making web application development simple. Its flagship project is Atmo, a platform that integrates three underlying projects also built by Suborbital. Vektor allows developers to write self-contained functions called Runnables to handle business logic, which are then built into WebAssembly. Atmo then automatically scales out a flat network of instances to handle traffic using Grav, a meshed message bus, and Reactr, an embedded job scheduler. Together, these projects make it simple to create powerful, WebAssembly-based server applications without worrying about infrastructure or writing boilerplate code. Connor Hicks maintains the Suborbital open source project. By day, he is a developer and Product Discovery Lead at 1Password. He joins the show today to talk about the exciting potential of WebAssembly, how Atmo is introducing new design patterns, and why we should think differently about complex webservices systems. Sponsorship inquiries: [email protected]

Observability is a key feature of a well-architected application. Because building an observability system for a cloud application can be challenging, especially at scale, many organizations elect to use third-party observability platforms rather than build internal tools. But these third-party provider contracts often charge by volume of data collected, which can be unpredictable and difficult to control. Organizations seeking to make their systems observable faced a tradeoff between convenience and control. Opstrace is building an observability platform that seeks to circumvent that tradeoff by matching costs to the users who get value from their service. Opstrace’s platform is open-source, and its clusters can be created easily via command line tools. Opstrace clusters talk directly to your cloud provider, and can store your data safely and inexpensively in S3 or GCS buckets. Sebastien Pahl is the co-founder and CEO of Opstrace. Prior to founding Opstrace, he was a Director of Engineering at Red Hat and Mesosphere. Sebastien joins the show today to talk about the evolution of the market for cloud observability tools, how Opstrace helps bring observability costs under control, and what the future has in store for the Opstrace open-source project. Sponsorship inquiries: [email protected]

As the volume and scope of data collected by an organization grow, tasks such as data discovery and data management grow in complexity. Simply put, the more data there is, the harder it is for users such as data analysts to find what they’re looking for. A metadata hub helps manage Big Data by providing metadata search and discovery tools, and a centralized hub which presents a holistic view of the data ecosystem. DataHub is Linkedin’s open-sourced metadata search and discovery tool. It is Linkedin’s second generation of metadata hubs after WhereHows. Pardhu Gunnam and Mars Lan join us today from Metaphor, a company they co-founded to build out the DataHub ecosystem. Pardhu and Mars, and the other co-founders of Metaphor, were part of the team at Linkedin that built the DataHub project. They join the show today to talk about how DataHub democratizes data access for an organization, why the new DataHub architecture was critical to Linkedin’s growth, and what we can expect to see from the DataHub project moving forwards. Sponsorship inquiries: [email protected]

The shift to microservices architectures and distributed systems has been a challenge for systems using conventional security practices, such filtering IP addresses using network policies. In addition, the increasing intersection of development and operations exemplified by the DevOps methodology has expanded the scope responsibilities in implementing secure systems. Part of CNCF, SPIFFE is a set of open-source specifications for issuing identity to services in heterogenous, distributed environments such as a cloud-native microservices architecture. Systems implementing SPIFFE bypass the need for application-level authentication and network-level ACL configuration. SPIRE, or the SPIFFE Runtime Environment, is a system that implements the SPIFFE standards to manage platform and workload attestation, providing an API for controlling policies, and coordinating certificate issuance and rotation. Derek Edwards is the head of engineering at Anthem.ai, and Ryan Turner is a software engineer at Uber. They join the show today to talk about the challenges of managing security in a distributed system, how adopting SPIFFE represented a paradigm shift in their authentication workflow, and how the SPIFFE and SPIRE projects are evolving to meet the needs of the next generation of cloud-native applications. HPE sponsored this podcast. Sponsorship inquiries: [email protected]

Software-Defined Networking describes a category of technologies that separate the networking control plane from the forwarding plane. This enables more automated provisioning and policy-based management of network resources. Implementing software-defined networking is often the task of Site Reliability Engineers, or SREs. Site reliability engineers work at the intersection of development and operations by bringing software development practices to system administration. Equinix manages co-location data centers and provides networking, security, and cloud-related services to their clients. Equinix is leveraging its status as a market leader in on-prem networking capabilities to expand into cloud and IaaS offerings such as Equinix Metal, which has been referred to as “bare-metal-as-a-service,” and offers integrations with 3rd party cloud technologies with a goal of creating a seamless alternative to modern public clouds for organizations seeking the benefits of colocation. Tim Banks is a Principal Solutions Architect at Equinix and he joins the show to talk about what Equinix offers and how it differs from other cloud providers. Sponsorship inquiries: [email protected]

Google Cloud, AWS, and Azure are the dominant cloud providers on the market today. But the market is still highly competitive, and there is significant overlap in the services offered by all three large providers. Since all three offer a broad range of services, developers looking to choose a platform for their application must focus on providers’ domains of relative excellence and how those align with their needs. One domain where Google Cloud Platform excels is with its database offerings. Google has data management baked into its organizational DNA, and has been the source of several innovative technologies in the data space such as Spanner, BigTable, and BigQuery. Andi Gutmans is a general manager and VP of engineering for databases at Google. He joins the show today to talk about how Google came to excel at databases and data management, how machine learning and Big Data users in particular can benefit from Google Cloud’s offerings, and how new features such as Database Migration Service are helping Google stay ahead of the curve in a competitive cloud landscape. Sponsorship inquiries: [email protected]

Vectors are the foundational mathematical building blocks of Machine Learning. Machine Learning models must transform input data into vectors to perform their operations, creating what is known as a vector embedding. Since data is not stored in vector form, an ML application must perform significant work to transform data in different formats into a form that ML models can understand. This can be computationally intensive and hard to scale, especially for the high-dimensional vectors used in complex models. Pinecone is a managed database built specifically for working with vector data. Pinecone is serverless and API-driven, which means engineers and data scientists can focus on building their ML application or performing analysis without worrying about the underlying data infrastructure. Edo Liberty is the founder and CEO of Pinecone. Prior to Pinecone, he led the creation of Amazon SageMaker at AWS. He joins the show today to talk about the fundamental importance of vectors in machine learning, how Pinecone built a vector-centric database, and why data infrastructure improvements are key to unlocking the next generation of AI applications. Sponsorship inquiries: [email protected]

Email has become such a routine feature of knowledge work that we often take it- and the email clients we use for it- for granted. While advancements such as intelligent spam filtering have improved the experience, many email clients retain the same basic structure and offer a largely similar experience. Superhuman is building a modern email client meant to reimagine email from the ground up. Superhuman is built to be fast, and seamlessly integrates insights from social networks such as Linkedin. It offers features such as undo send, AI triage, and mail status tracking. Superhuman even works offline, using Service Workers to serve cached assets when a network connection is not available. Emuye Reynolds is the Head of Engineering at Superhuman. She was formerly a Senior Software Developer at Apple and led the development of the UI for Apple TV UI. She joins the show today to talk about what’s lacking from traditional email clients, what engineering challenges her team faces when building across multiple platforms, and how Superhuman’s new features represent an evolutionary step forward for email client technology. Sponsorship inquiries: [email protected]

WordPress is a free and open-source content management system, or CMS, written in PHP. Since its release in 2003, WordPress has become ubiquitous on the web. It is estimated that roughly 60 million websites use WordPress as a CMS. However, despite its popularity, Wordpress has limitations in its design. Wordpress sites are dynamic, and the front and back end are tightly coupled. A dynamic, full-stack application can be useful when handling complex functionality, but it also slows down the site and opens up security vulnerabilities. Strattic is a static site generator and hosting platform that specializes in converting WordPress sites into a static architecture. Static pages are isolated from the backing application, improving security against common Wordpress vulnerabilities. Modern web users have high expectations for speed and security, and Strattic helps WordPress sites achieve this without sacrificing the benefits of the Wordpress platform. Zeev Suraski is the CTO of Strattic. Zeev is one of the architects and principal authors of the PHP language, which is the foundation of WordPress. Zeev joins the show today to talk about the place of PHP in modern web development, and how Strattic helps Wordpress developers build modern, fast, and secure sites. Sponsorship inquiries: [email protected]

AWS offers over 200 services as part of its IaaS platform, and that number continues to grow. Organizing all of these services, and tracking the costs they incur, can be a significant challenge, often requiring teams of AWS-certified sysadmins working together to get a handle on an enterprise-scale system. Vantage provides an alternative, streamlined AWS console that makes it easier to manage AWS services and track associated costs. Users link their AWS account to Vantage, and it automatically profiles all their services and aggregates the information into a dashboard. Users can customize how their Vantage console appears and allows users to break down service usage by region. Ben Schaechter is the co-founder of Vantage. Before founding Vantage he was a Senior Product Manager at AWS and DigitalOcean. Ben joins the show today to talk about how Vantage helps streamline the AWS experience, and why teams of all sizes can benefit from a better user experience on the AWS platform. Sponsorship inquiries: [email protected]

A data warehouse is a centralized repository that an enterprise may use to store selected data from production systems. Data is transformed into a structured form that makes it easily accessible for business intelligence or other operational users. SQL-compliant databases are frequently used for data warehouses due to the popularity of SQL as a tool in business data analytics. PostgreSQL is a free and open-source relational database management system. Postgres-based databases are widespread and are used by a variety of organizations, from Reddit to the International Space Station, and Postgres databases are a common offering from cloud providers such as AWS, Alibaba Cloud, and Heroku. Josh Drake and Thomas Richter are experts on Postgres data warehousing. They join the show today to talk about the staying power of Postgres, why Postgres is a good choice for data warehousing, and how cloud technology is changing relational database management systems. Sponsorship inquiries: [email protected]

Build automation tools automate the process of building code, including steps such as compiling, packaging binary code, and running automated tests. Because of this, build automation tools are considered a key part of a continuous delivery pipeline. Build automation tools read build scripts to define how they should perform a build. Common build scripts include Makefile, Dockerfile, and bash. Earthly is a build automation tool that allows you to execute all your builds in containers. Earthly uses Earthfiles, which draws from the best features of Makefile and Dockerfile and provides a common layer between language-specific tooling and the CI build spec. Earthly builds are repeatable, isolated, and self-contained, and will run the same way across different environments such as a CI system or a developer’s laptop. Vlad Ionescu is the Founder and CEO of Earthly Technologies. He was formerly the founder and chief architect at ShiftLeft.io. Vlad joins the show today to talk about why reproducible builds are important, how Earthly simplifies build scripts, and what the long-term vision for Earthly looks like. Sponsorship inquiries: [email protected]

Static analysis is a type of debugging that identifies defects without running the code. Static analysis tools can be especially useful for enforcing security policies by analyzing code for security vulnerabilities early in the development process, allowing teams to rapidly address potential issues and conform to best practices. R2C has developed a fast, open-source static analysis tool called Semgrep. Semgrep provides syntax-aware code scanning and a database of thousands of community-defined rules to compare your code against. Semgrep also makes it easy for security engineers and developers to define custom rules to enforce their organization’s policies. R2C’s platform has been adopted by industry leaders such as Dropbox and Snowflake, and recently received the “Disruptive Innovator” distinction at Forbes’ 2020 Cybersecurity Awards. Isaac Evans is the Founder and CEO of R2C. Before founding R2C he was an Entrepreneur in Residence at Redpoint Ventures and a computer scientist at the US Department of Defense. Isaac joins the show today to talk about how R2C is helping teams improve their cloud security, why static analysis is a natural fit for CI/CD workflows, and what to expect from R2C and the Semgrep project in the future. Sponsorship inquiries: [email protected]

Modern SaaS products are increasingly delivered via the cloud, rather than as downloadable, executable programs. However, many potential users of those SaaS products may need that software deployed on-prem, in a private network. Organizations have a variety of reasons for preferring on-prem software, such as security, integration with private tools, and compliance with regulations. The cost of setting up a bespoke on-prem version of a SaaS offering was often prohibitive for both the vendor and potential users. Replicated leverages the portability of containers to help SaaS vendors ship an on-prem or multi-prem version of their software. Replicated gives SaaS vendors a suite of ready-made components to help install and manage an instance of their software on-prem. Replicated has seen explosive growth in its six-year lifespan: today, 50 of the Fortune 100 companies manage apps with Replicated. Grant Miller is the Founder and CEO of Replicated. Replicated recently closed a Series B fundraising round to help scale their work in the cloud-native space, including the launch of their Kubernetes-Off-the-Shelf platform. He joins the show today to talk about the next generation of tools on the Replicated platform, how Kubernetes is changing enterprise IT, and why the on-prem software market isn’t going away anytime soon. Sponsorship inquiries: [email protected]

Cloud platforms are often categorized as providing either Infrastructure-as-a-Service or Platform-as-a-Service. On one side of the spectrum are IaaS giants such as AWS, which provide a broad range of services for building infrastructure. On the other are PaaS providers such as Heroku and Netlify which abstract away the lower-level choices and focus on developer experience. Digital Ocean has carved out a sizable niche in the cloud hosting space by targeting the middle ground- a streamlined cloud platform built for developers, which still offers the ability to choose, customize, and manage infrastructure. The release of Digital Ocean’s App Platform takes this goal a step further. The App Platform allows users to build and deploy an app or static site directly from GitHub directly onto a DigitalOcean-managed Kubernetes cluster. Teams can access the power, scale, and flexibility of Kubernetes without having to worry about the complexity of managing a cluster themselves. The App Platform gives developers the choice of how much of their infrastructure they want to control, and how much they want to be provided by the platform. Cody Baker and Apurva Joshi work at Digital Ocean. They join the show today to talk about why Digital Ocean stands out in a competitive cloud hosting space, what is the value proposition for developers interested in the App Platform, and how the PaaS industry is evolving. Sponsorship inquiries: [email protected]

Yelp.com is a crowdsourced review platform focused on restaurants and local businesses. Originally created as an email-based recommendation service, Yelp re-launched in its modern form in 2005. At the time, its focus on user-created reviews and social interactions was fairly novel, and made it stand out from competitors such as Angie’s List and CitySearch. Since then, Yelp has become a worldwide brand, and as of 2021 it has over 171 million reviews on its site. The mid- to-late 2000s represented a time of explosive growth and profound change in the web application space. Industry leaders like Yelp had to adapt their technology stacks for the unprecedented scaling they were experiencing. At the same time, the rise of smartphones led Yelp and many others into the mobile application space. Michael Stoppelman was an engineer at Yelp during this turbulent time. He left Yelp in 2015, and now works as an angel investor. He joins the show today to talk about the engineering challenges Yelp’s team faced during this time, the profound changes that the industry as a whole went through, and how the history of Yelp can help us contextualize the startup landscape today. Sponsorship inquiries: [email protected]

Studies show that people in “maker” professions such as developers and writers are most productive when they can carve out dedicated time for focused work, without the frequent context-switching that comes with an irregular meeting schedule. Meetings and other non-development work are necessary parts of the job, but a team will be much more productive with deep work time in mind. Okay is an engineering metrics dashboard platform designed with the goal of maximizing time for deep work. Okay helps break down time slots into categories such as Maker Time, Meeting Load, and Friction Time based on data collected and feedback from the team. Okay organizes both quantitative and qualitative data into a single dashboard for team planning, and supports plug-and-play integrations with productivity tools such as GoogleCalendar, PagerDuty, and CircleCI, and more. Tomas Barreto is the CTO and co-founder of Okay. Before founding Okay, he worked with Sequoia Capital and Y Combinator, and was VP of engineering at Box. He joins the show today to talk about why Maker Time is so important for engineers, and how Okay helps teams make data-driven decisions to maximize productivity. Sponsorship inquiries: [email protected]

A “co-location” center is a data center that leases out networking and compute infrastructure to retail clients. Co-location centers host clients with a wide variety of infrastructure strategies, from small retail customers, to medium-size teams running hybrid cloud models, to large corporate clients who prefer not to incur the capital cost of building their own data center. While Equinix is already a market leader in co-location centers, they have expanded to provide a wide variety of services for their clients, including managed IaaS, disaster recovery, and integrations with cloud providers such as AWS and Google Cloud. Shaedon Blackman is a partner developer analyst at Equinix. As a partner developer analyst, Shaedon works to build Equinix’s network of corporate partners, while also advocating for diverse and inclusive human capital within the organization. Before he joined Equinix, he was a Core Fellow at Pursuit, a software engineering fellowship funded by Google, and the Chief Operating Officer of a non-profit youth program. He joins the show today to talk about a variety of important topics facing the tech industry today, including diversity, inclusion, and education, and also how Equinix is building partnerships and sponsoring open-source projects to achieve its goals. Sponsorship inquiries: [email protected]

Prediction Markets provide an exchange for trading based on the outcome of events. Most prediction markets are centralized- they operate like a casino, where betting takes place under the supervision of one central governing organization. This makes the market less efficient than it otherwise might be: the central organization is a business, and it makes money by extracting value from the trades the customers make. Augur is a prediction market built on the Ethereum blockchain. A trading network built on a blockchain can have a decentralized, permissionless transaction record without a centralized, governing body. Augur’s network is built to be transparent, low-cost, and free from interference. Joey Krug joins us today from Pantera Capital, a venture capital fund focused on Blockchain technology. Joey is also a co-founder of the Forecast Foundation, which contributes to the development of the Augur open-source project. We discuss what it takes to build a trustworthy decentralized market, how Augur is solving challenges such as the oracle problem, and why blockchain may be the key to democratizing finance. Sponsorship inquiries: [email protected]

In the past several years, Kubernetes has become the de-facto standard for orchestrating containerized, stateless applications. Tools such as StatefulSets and Persistent Volumes have helped developers build stateful applications on Kubernetes, but this can quickly become difficult to manage as an application scales. Tasks such as machine learning, distributed AI, and big data analytics often require a distributed application to maintain some sort of state across services. KubeDirector is an open-source controller that helps streamline the deployment and management of complex stateful scale-out application clusters on Kubernetes. KubeDirector provides an application-agnostic deployment pattern and enables developers to run non-cloud native stateful applications on Kubernetes without modifying the code. KubeDirector aims to bring enterprise-level capabilities for distributed stateful applications to Kubernetes. Kartik Mathur is an engineer at HPE Developer, an open-source initiative within Hewlett-Packard Enterprise. HPE is an enterprise contributor to the KubeDirector open-source community. Kartik previously worked as senior software engineers at BlueData, which created the KubeDirector project before its acquisition by HPE. Kartik joins the show today to talk about why state is important for Big Data or Machine learning applications, how KubeDirector can help manage the complexity of stateful applications. Sponsorship inquiries: [email protected]

Over the past few years, the conventional wisdom around the value proposition of Big Data has begun to shift. While the prevailing attitude towards Big Data may once have been “bigger is better,” many organizations today recognize that broad-scale data collection comes with its own set of risks. Data privacy is becoming a hotly debated topic both in the technology industry and in regulatory agencies and governments. Bigger and less private datasets are more attractive targets for hackers, meaning that an organization must invest heavily in security as well to avoid a breach. Every organization faces a tradeoff between the value of the insights produced from large datasets versus increased storage costs and increasing privacy risks. Tonic is building a “synthetic data” platform to address these tradeoffs and help organizations mitigate data risk. Tonic takes in raw data, perhaps from a data lake, and transforms it into more manageable, de-identified data sets for ease of use and user privacy. Tonic can create statistically identical, structured datasets that allow software engineers and business analysts to extract the same useful insights that drive an organization’s progress, without the risk of working with identifiable, private user data. Ian Coe, Andrew Colombi, and Adam Kamor are co-founders of Tonic. Along with their fourth co-founder, Karl Hanson, Ian, Andrew, and Adam all worked together at Palantir Technologies where the idea for Tonic was born. They join the show today to talk about the value of synthetic data, the risks and rewards of big data, and how compliance, privacy, and security are driving innovation in the data management sector. Sponsorship inquiries: [email protected]

Serverless computing refers to an architectural pattern where server-side code is run on-demand by cloud providers, who also handle server resource allocation and operations. Of course, there is a server involved on the provider’s side, but administrative functions to manage that server such as capacity planning, configuration, or management of containers are handled behind-the-scenes, allowing the application developers to focus on business logic. This makes for highly elastic and scalable systems and can reduce development, testing, and iteration time due to reduced overhead. Function as a Service (FaaS) describes a model of serverless computing where services are decomposed into modular functions and deployed to a serverless platform. These functions are executed only when called and are typically stateless. Despite the benefits of elasticity and modularity that FaaS offers, it has drawbacks as well. Taking disaggregation of functionality to an extreme means that behavior that formerly required a method call now may require a network call to another function, increasing latency and making larger-scale operations inefficient. Cloudburst is a stateful FaaS platform built to combine the power of low-latency mutable state and communication in Python with the elasticity and scalability allowed by serverless architecture. Johann Schleier-Smith is an entrepreneur and engineer who is currently a board member of Sama. Johann was formerly the founder and CTO of if(we), a social network and incubator. He is the co-author of the paper Cloudburst: Stateful Functions as a Service, and joins the show today to talk about how Cloudburst addresses the drawbacks of current FaaS models, and what’s next for serverless computing. Sponsorship inquiries: [email protected]

Blockchain technology has a wide variety of potential applications. Fields such as finance, supply chain management, and even voting have seen innovations driven by the development of distributed applications built on blockchains, called DApps. However, developing a DApp on a blockchain often requires low-level knowledge about cryptographic protocols or particular networks. Since no one blockchain platform has emerged as dominant- and the field itself is rapidly evolving- there is a high opportunity cost for developers if they choose to invest significant time learning one blockchain paradigm or another. Reach provides a platform for developing DApps, complete with a high-level language based on Javascript. Reach allows developers to write one set of code to specify the DApp and all its components, and which can be deployed onto any blockchain implementation under the hood. Reach’s goal is to allow developers to focus on writing business logic for their DApps rather than worrying about low-level implementation details and aims to smooth the steep learning curve for developers new to the world of blockchain. Chris Swenor and Jay McCarthy are the founders of Reach. Chris was formerly the co-founder and CEO of Alacris Protocol, an operating system for blockchain applications, and he is currently a technologist in residence and mentor at Harvard. Jay has been a computer science professor for over a decade, and worked on the development of the Racket programming language. Chris and Jay join the show today to talk about the challenges of developing on blockchain, how Reach helps make blockchain developers more productive, and how the blockchain ecosystem might evolve in the future.

The incredible advances in machine learning research in recent years often take time to propagate out into usage in the field. One reason for this is that such “state-of-the-art” results for machine learning performance rely on the use of handwritten, idiosyncratic optimizations for specific hardware models or operating contexts. When developers are building ML-powered systems to deploy in the cloud and at the edge, their goals to ensure the model delivers the best possible functionality and end-user experience- and importantly, their hardware and software stack may require different optimizations to achieve that goal. OctoML provides a SaaS product called the Octomizer to help developers and AIOps teams deploy ML models most efficiently on any hardware, in any context. The Octomizer deploys its own ML models to analyze your model topology, and optimize, benchmark, and package the model for deployment. The Octomizer generates insights about model performance over different hardware stacks and helps you choose the deployment format that works best for your organization. Luis Ceze is the Co-Founder and CEO of OctoML. Luis is a founder of the ApacheTVM project, which is the basis for OctoML’s technology. He is also a professor of Computer Science at the University of Washington. Jason Knight is co-founder and CPO at OctoML. Luis and Jason join the show today to talk about how OctoML is automating deep learning engineering, why it’s so important to consider hardware when building deep learning systems, and how the field of deep learning is evolving. Sponsorship inquiries: [email protected]

A data-driven organization collects a wide variety of data to help in strategic decision-making. The cost of storing large amounts and variety of data has dropped dramatically in the last two decades, but too much unstructured data may not improve decision-making, and can even lead to “analysis paralysis.” Organizations react by extracting the most important, actionable data and placing it into a data warehouse, which has a predesigned structure meant to streamline the data in preparation for analysis. The key challenge with this approach is identifying what should be streamlined, and how to structure the data warehouse to focus on the most important, actionable items. This is especially important for organizations seeking to scale, as the necessary structure to generate the most relevant insights may change as the organization grows. Narrator is building data intelligence that uses a simple, proprietary Universal Data Model to help organizations streamline their data warehousing. Narrator is built on the belief that data tells the story of a system, and its platform empowers organizations to use those stories to make better decisions. Ahmed Elsamadisi is the founder and CEO of Narrator. Before founding Narrator, he spent several years working in data analysis and algorithm design for WeWork, Raytheon, and Cornell’s Autonomous Systems Laboratory. He joins the show today to talk about how Narrator generates the most actionable insights from a data warehouse, why a Universal Data Model is so important when scaling, and what makes Narrator’s approach to data analysis different. Sponsorship inquiries: [email protected]

Open source software is software distributed along with its source code, using a permissive license that allows anyone to view, use, or modify it. The term “open source” also refers more broadly to a philosophy of technology development which prioritizes transparency and community development of a project. Typically, development is managed by a governing body, whether a company, foundation, or just a group of passionate users, and work is done in public repositories like Github. Nearly every corner of the software engineering world has been impacted in some way by open source. Well-known open source projects include Linux, Kubernetes, and Wordpress. Kevin Xu is the author of Interconnected, a bilingual newsletter on tech, business, and U.S-China relations. He is an investor in open source startups at OSS Capital, and formerly served in the Obama White House. He joins the show today to talk about the benefits of open source in the public and private sectors, and how open source will be critical to the development of high-tech industry in our country as we pivot to facing some of the 21st century’s most pressing challenges. Sponsorship inquiries: [email protected]

Video calling over the internet has experienced explosive growth in the last decade. In 2010, surveys estimated that around 1 in 5 Americans had tried online video calling for any reason. By May of 2020, that number had nearly tripled. A significant factor in the growth of video calling has been an open-source project called WebRTC, or “Web Real-Time Communication.” WebRTC makes it possible to capture and stream audio or video data between browsers without the use of plugins or third-party software. Daily is a developer platform that builds on WebRTC to provide realtime video APIs for developers. Developers can easily add video call widgets to their code which come with a set of default configurations for functions such as bandwidth management and cross-browser support. Daily also offers a set of frontend libraries and REST APIs for developers who want to build a customized experience. Kwindla Hultman Kramer is a co-Founder at Daily, and he’s joined today by Wesley Faulkner, who handles developer relations. They join the show today to talk about the growth in demand for video calling services, building a developer-friendly video calling API, and what’s next for video calling applications. Sponsorship inquiries: [email protected]

Cilium is open-source software built to provide improved networking and security controls for Linux systems operating in containerized environments along with technologies like Kubernetes. In a containerized environment, traditional Layer 3 and Layer 4 networking and security controls based on IP addresses and ports, like firewalls, can be difficult to operate at scale because of the volatility of the system. Cilium is eBPF, which is an in-kernel virtual machine which attaches applications directly to code paths in the kernel. In effect, this makes the Linux kernel “programmable” without changing kernel source code or loading modules. Cilium takes advantage of this functionality to insert networking and security functions at the kernel level rather than in traditional Layer 3 or Layer 4 controls. This allows Cilium to combine metadata from Layer 3 and Layer 4 with application-layer metadata such as HTTP method and header values in order to establish rules and provide visibility based on service, pod, or container identity. Isovalent, co-founded by the creator of Cilium, maintains the Cilium Open Source Project and also offers Cilium Enterprise, which is a suite of tools helping organizations adopt Cilium and overcome the hurdles of building a secure, stable cloud-native application. Dan Wendlant and Thomas Graf are the co-founders of Isovalent. Thomas, the firm’s CTO, was the original creator of the Cilium open-source project and spent 15 years working on the Linux kernel prior to founding Isovalent. Dan, Isovalent’s CEO, has also worked at VMWare and Nicira. They join the show today to talk about why Cilium and Cilium Enterprise are a great choice for organizations looking to build cloud-native applications. Sponsorship inquiries: [email protected]

In a distributed application, observability is key to handling incidents and building better, more stable software. Legacy monitoring methods were built to respond to predictable failure modes, and to aggregate high-level data like access speed, connectivity, and downtime. Observability, on the other hand, is a measure of how well you can infer the internal state of a system from its outputs in order to trace the cause. At its core, building a system with observability means using instrumentation to provide insights on how and why internal components within a system are performing a certain way. Developers and SREs can build on that data to proactively debug potential failure modes, set service-level objectives, and speed up incident response. New Relic has been an industry leader in the observability space for the better part of a decade. This year, they announced New Relic One, an evolution of their flagship platform that streamlines and simplifies all the functions available to help organizations achieve observability. New Relic One enhances the Full-Stack Observability Platform through AIOps with their Applied Intelligence, which draws insights from the observability data to help detect anomalies before they become incidents. Lew Cirne is the founder and CEO of New Relic. He joins the show today to talk about how New Relic One helps developers move beyond monitoring and embrace observability, and how he sees the future of software observability platforms. Sponsorship inquiries: [email protected]

Embedded Software Engineering is the practice of building software that controls embedded systems- that is, machines or devices other than standard computers. Embedded systems appear in a variety of applications, from small microcontrollers, to consumer electronics, to large-scale machines such as cars, airplanes, and machine tools. iRobot is a consumer robotics company that applies embedded engineering to build robots that perform common household tasks. Its flagship product is the Roomba, perhaps one of the most well-known autonomous consumer robots on the market today. iRobot’s engineers work at the intersection of software and hardware, and work in a variety of domains from electrical engineering to AI. Chris Svec is a Software Engineering Manager at iRobot. He started his career designing x86 chips and later moved up the hardware/software stack into embedded software. He joins the show today to talk about iRobot, the design process for embedded systems, and the future of embedded systems programming. Sponsorship inquiries: [email protected]

Security is more important than ever, especially in regulated fields such as healthcare and financial services. Developers working in highly regulated industries often spend considerable time building tooling to help improve compliance and pass security audits. While the core of many security workflows is similar, each industry and each organization may have its own idiosyncratic needs or particular regulatory requirements to meet. Sym is a platform for building security workflows that seeks to build on those core similarities while empowering developers with the tools they need to meet their application’s unique security and compliance needs. Sym believes in putting engineers in control of security, in the same way that DevOps put engineers in control of infrastructure. Yasyf Mohamedali is the CEO and co-founder of SymOps. Before SymOps, he was the CTO of Karuna Health. He joins the show today to talk about security and innovation in regulated industries and how Sym can help developers close the intent-to-implementation gap in application security. Sponsorship inquiries: [email protected]

Microservices are built to scale. But as a microservices-based system grows, so does the operational overhead to manage it. Even the most senior engineers can’t be familiar with every detail of dozens- perhaps hundreds- of services. While smaller teams may track information about their microservices via spreadsheets, wikis, or other more traditional documentation, these methods often prove unsuitable for the unique demands of a sprawling microservices system. A microservices catalog is a solution to this problem. A microservices catalog seeks to centralize information about the services in your software architecture, including the purpose of a service, its owner, and instructions for using it. A microservices catalog can also provide a centralized source of knowledge about a system, which can help on-call engineers diagnose issues and also provide resources for onboarding new team members. Larger companies sometimes devote significant internal resources toward developing in-house microservices catalogs, while smaller organizations may not have the resources at their disposal to do so. OpsLevel’s founders recognized that many teams were re-inventing the wheel building internal microservices catalogs, and set out to design a toolset that could meet the needs of users of all sizes. OpsLevel’s team has drawn from extensive experience working with industry leaders in DevOps to create a comprehensive toolset for managing microservices infrastructure. OpsLevel provides a “single pane of glass for operations,” integrating with a variety of tools such as Slack, git, CI/CD, incident management, and deployment systems. John Laban and Kenneth Rose are the co-founders of OpsLevel. Before John and Kenneth founded OpsLevel they worked together at PagerDuty, where John was the first engineer on the team. Kenneth, OpsLevel’s CTO, was also previously a senior developer at Shopify. John and Kenneth join the show today to talk about how OpsLevel can help developers manage their microservices better, and even transform how their team does DevOps. Sponsorship inquiries: [email protected]

Reinforcement learning is a paradigm in machine learning that uses incentives- or “reinforcement”- to drive learning. The learner is conceptualized as an intelligent agent working within a system of rewards and penalties in order to solve a novel problem. The agent is designed to maximize rewards while pursuing a solution by trial-and-error. Programming a system to respond to the complex and unpredictable “real world” is one of the principal challenges in robotics engineering. One field which is finding new applications for reinforcement learning is the study of MEMS devices- robots or other electronic devices built at the micrometer scale. The use of reinforcement learning in microscopic devices poses a challenging engineering problem, due to constraints with power usage and computational power. Nathan Lambert is a PhD student at Berkeley who works with the Berkeley Autonomous Microsystems Lab. He has also worked at Facebook AI Research and Tesla. He joins the show today to talk about the application of reinforcement learning to robotics and how deep learning is changing the MEMS device landscape. Sponsorship inquiries: [email protected]

For startups that are still seeking product/market fit, pre-seed investments are critical to funding initial investments in the product and in the infrastructure needed to scale. Afore Capital is a pre-seed fund that invests in innovative companies across a wide variety of verticals. Afore focuses on startups with unique product insights and novel distribution approaches. Gaurav Jain is a co-Founder and Managing Partner at Afore Capital. Before that, he worked with the Android team at Google. Gaurav joins the show today to talk about the risks and rewards of pre-seed investing, and about how founders and investors can find opportunity in the current venture environment. Sponsorship inquiries: [email protected]

Kafka has achieved widespread popularity as a popular distributed queue and event streaming platform, with enterprise adoption and a billion dollar company (Confluent) built around it. But could there be value in building a new platform from scratch? Redpanda is a streaming platform built to be compatible with Kafka, that does not require the JVM nor Zookeeper, both of which are dependencies that made Kafka harder to work with than perhaps necessary. Alexander Gallego is a core committer to Redpanda and joins the show to talk about why he started the project and its value proposition. Sponsorship inquiries: [email protected]

Companies can have a negative impact on the environment by outputting excess carbon. Many companies want to reduce their net carbon impact to zero, which can be done by investing in forests. Pachama is a marketplace for forest investments. Pachama uses satellites, imaging, machine learning, and other techniques to determine how much carbon is being absorbed by different forests. Diego Saez-Gil is a founder of Pachama, and joins the show to talk through how Pachama works and the long-term goals of the company. Sponsorship inquiries: [email protected]

Rust and Golang are two of the newest lower level languages for doing systems programming. They are often used for applications such as file systems, operating systems, and latency-sensitive applications. How do they compare in terms of safety, speed, and programming ergonomics? Linhai Song is an assistant professor and researcher at Penn State University, and joins the show to talk about his work researching Go and Rust. Sponsorship inquiries: [email protected]

GraphQL has changed the common design patterns for the interface between backend and frontend. This is usually achieved by the presence of a GraphQL server, which interprets and federates a query from the frontend to the backend server infrastructure. Dgraph is a distributed graph database with native GraphQL support. Manish Jain is a founder of Dgraph, and joins the show to talk about its purpose and his vision for the future of the technology. Sponsorship inquiries: [email protected]

Agriculture infrastructure allows plants such as corn, soy, and wheat to move from large scale farms to consumers all around the world. The relevant players in the agricultural infrastructure includes growers, shippers, and planners. These individuals need new technology to interact more efficiently. Growers need to be able to connect more smoothly with buyers. Farmers need better management of their carbon credits. Microbial technology can allow plants to be better shielded from tough conditions. Agricultural health, transport, commerce, and logistics are all problems that Indigo Agriculture is focused on solving. David Potere is head of geoinnovation at Indigo, and joins the show to talk about the problems the company is solving and the engineering practices at Indigo. Sponsorship inquiries: [email protected]

Network discovery allows enterprises to identify what devices are on their network. These devices can include smartphones, servers, desktop computers, and tablets. Being able to index the devices on a network is crucial to figuring out the security profile of that network. HD Moore is a founder of Rumble Networks, a company focused on network discovery and asset inventory. He joins the show to talk about how network discovery works and his experience building Rumble. Sponsorship inquiries: [email protected]

Large technology companies are a new type of industry. Their power and reach is resistant to a comparison of previous generations, such as big oil. Alex Kantrowitz is a journalist who has covered big technology for much of his career, and he currently runs Big Technology, a newsletter and podcast about the biggest technology companies in the world. He’s also the author of “Always Day One: How The Tech Titans Plan To Stay On Top Forever”. Alex joins the show to talk about his work and share his thoughts on big tech.

Cost management is growing in importance for companies that want to manage their significant cloud bill. Kubernetes plays an increasing role in modern infrastructure, so managing cost of Kubernetes clusters becomes important as well. Kubecost is a company focused on giving visibility into Kubernetes resources and reducing spend. Webb Brown is a founder of Kubecost and joins the show to talk about Kubernetes cost optimization and what he is building with Kubecost. Sponsorship inquiries: [email protected]

TensorFlow Lite is an open source deep learning framework for on-device inference. TensorFlow Lite was designed to improve the viability of machine learning applications on phones, sensors, and other IoT devices. Pete Warden works on TensorFlow Lite at Google and joins the show to talk about the world of machine learning applications and the necessary frameworks and devices necessary to build them. Sponsorship inquiries: [email protected]

Data lakes and data warehouses store high volumes of multidimensional data. Data sources for these pieces of infrastructure can become unreliable for a variety of reasons. When data sources break, it can cause downstream problems. One company working to solve the problem of data reliability is Monte Carlo Data. Barr Moses and Lior Gavish are founders of Monte Carlo and join the show to talk about data reliability and the overall landscape of data infrastructure. Sponsorship inquiries: [email protected]

When Tim Wagner worked at Amazon, he invented AWS Lambda. After working on the early serverless infrastructure, he joined Coinbase and worked as VP of Engineering. Since leaving Coinbase, he has started a new company called Vendia. Vendia combines his learnings from the serverless space with the innovations around blockchains to work on the problem of data sharing. Tim and David Wells join the show to discuss what they are working on with Vendia. Sponsorship inquiries: [email protected]

Serverless has grown in popularity over the last five years, and the space of applications that can be built entirely with serverless has increased dramatically. This is due to two factors: the growing array of serverless tools (such as edge-located key value stores) and the rising number of companies with serverless offerings. One of those companies is Fastly, which originally gained adoption for its CDN solution. Tyler McMullen is the CTO of Fastly and he joins the show to talk through how Fastly looks at edge computing today. This is Tyler’s third appearance on the show. Sponsorship inquiries: [email protected]

Originally published July 30, 2019 “Internet of Things” is a term used to describe the increasing connectivity and intelligence of physical objects within our lives. IoT has manifested within enterprises under the term “Industrial IoT,” as wireless connectivity and machine learning have started to improve devices such as centrifuges, conveyor belts, and factory robotics. In the consumer space, IoT has moved slower than many people expected, and it remains to be seen when we will have widespread computation within consumer devices such as microwaves, washing machines, and lightswitches. IoT computers have different constraints than general purpose computers. Security, reliability, battery life, power consumption, and cost structures are very different in IoT devices than in your laptop or smartphone. One technology that could solve some of the problems within IoT is WebAssembly, a newer binary instruction format for executable programs. Jonathan Beri is a software engineer and the organizer of the San Francisco WebAssembly Meetup. He has significant experience in the IoT industry, and joins the show to discuss the state of WebAssembly, the surrounding technologies, and their impact on IoT. Sponsorship inquiries: [email protected]

Originally published July 19, 2019 In 2011, Facebook had begun to focus its efforts on mobile development. Mobile phones did not have access to reliable, high bandwidth connections, and the Facebook engineering team needed to find a solution to improve the request latency between mobile clients and the backend Facebook infrastructure. One source of latency was recursive data fetching. If a mobile application client made a request to the backend for newsfeed, the backend API would return the newsfeed, but some components of that feed would require additional requests to the backend. In practice, this might result in a newsfeed loading partially on a phone, but having a delayed loading time for the comments of a newsfeed item. GraphQL is a solution that came out of this problem of recursive data fetching. A GraphQL server provides middleware to aggregate all of the necessary information to serve a complete request. GraphQL connects to backend data sources and federates the frontend request across these different data sources. GraphQL was open sourced in 2015, and has found many use cases in addition to simplifying backend data fetching for mobile clients. Today, GraphQL is used by PayPal, Shopify, Twitter, and hundreds of other companies. Lee Byron is the co-creator of GraphQL and he joins the show to tell the story of GraphQL, and how it fit into Facebook’s shift to mobile. Sponsorship inquiries: [email protected]

Originally published September 13, 2019 Amazon Web Services first came out in 2006. It took several years before the software industry realized that cloud computing was a transformative piece of technology. Initially, the common perspective around cloud computing was that it was a useful tool for startups, but would not be a smart option for large, established businesses. Cloud computing was not considered economical nor secure. Today, that has changed. Every company that writes software is figuring out how to utilize the cloud. Software companies with on-prem servers are migrating old applications to the cloud, and most companies that have started in the last decade do not even have physical servers. Applications that are started on the cloud are referred to as “cloud-native.” The architecture of cloud-native applications is a newer topic of discussion, and some software patterns that became established in the pre-cloud era make less sense today. Cornelia Davis is VP of technology at Pivotal and the author of Cloud Native Patterns, a book about developing applications in the distributed, virtual world of the cloud. Cornelia was previously on the show to discuss Cloud Foundry. In today’s episode, our conversation centers on her book, and her perspective on the emerging patterns of cloud native software. Sponsorship inquiries: [email protected]

Originally published May 29, 2020 Kubernetes has become a highly usable platform for deploying and managing distributed systems. The user experience for Kubernetes is great, but is still not as simple as a full-on serverless implementation--at least, that has been a long-held assumption. Why would you manage your own infrastructure, even if it is Kubernetes? Why not use autoscaling Lambda functions and other infrastructure-as-a-service products? Matt Ward is a listener of the show and an engineer at Mux, a company that makes video streaming APIs. He sent me an email that said Mux has been having success with self-managed Kubernetes infrastructure, which they deliberately opted for over a serverless deployment. I wanted to know more about what shaped this decision to opt for self-managed infrastructure, and the costs and benefits that Mux has accrued as a result. Matt joins the show to talk through his work at Mux, and the architectural impact of opting for Kubernetes instead of fully managed serverless infrastructure. Sponsorship inquiries: [email protected]