Software Engineering Daily

Originally published July 14, 2016 EconTalk is a weekly economics podcast that has been going for a decade. On EconTalk, Russ Roberts brings on writers, intellectuals, and entrepreneurs for engaging conversations about the world as seen through the lens of economics. Russ Roberts is today’s guest, and it is a treat because I have been listening to EconTalk since 2006 and it was a central point of inspiration for what Software Engineering Daily has become. On this episode, we talk about how software impacts the world economically, from bitcoin’s promise of zero cost transactions to the opportunities and regulatory challenges of the software-enabled gig economy.

Originally published May 24, 2018 When a user takes a ride on Uber, the app on the user’s phone is communicating with Uber’s backend infrastructure, which is writing to a database that maintains the state of that user’s activity. This database is known as a transactional database or “OLTP” (online transaction processing). Every active user and driver and UberEATS restaurant is writing data to the transactional data store. Periodically, that data is copied from the transactional data system to a different data storage system, where that data can be queried for large-scale data analysis. For example, if a data scientist at Uber wants to get the average amount of miles that a given user rode in February, that data scientist would issue a query to the analytical data cluster. Uber uses the Hadoop distributed file system (HDFS) to store analytical data. On this file system, Uber has a version history of all of the company’s useful historical data. Trip history, rider activity, driver activity–every data point that is in the transactional database–but in a file format that is easier to query for large scale processing. This file format is known as Parquet. Data scientists, machine learning engineers, and real-time application developers all depend on the massive quantities of data that are stored in these Parquet files on Uber’s HDFS cluster. To simplify the access of that data by many different clients, Uber uses Presto, an analytical query engine originally built at Facebook. Presto translates SQL queries into whatever query language is necessary to access the underlying storage medium–whether that storage system is an ElasticSearch cluster, a set of Parquet files, or a relational database. Presto is useful because it simplifies the relationship between data engineers and the application developers who are building on top of the data engineering infrastructure. In today’s show, Zhenxiao Luo joins to give an end-to-end description of Uber’s data infrastructure–from the ingest point of the OLTP database to the OLAP data storage system on HDFS, to the wide range of data systems and applications that run on top of that OLAP data.

Originally published January 4, 2019 When a user makes a request to product like The New York Times, that request hits an API gateway. An API gateway is the entry point for an external request. An API gateway serves several purposes: authentication, security, routing, load balancing, and logging. API gateways have grown in popularity as applications have become more distributed, and companies offer a wider variety of services. If an API is public, and anyone can access it, you might need to apply rate limiting so that users cannot spam the API. If the API is private, the user needs to be authenticated before the request is fulfilled. Kong is a company that builds infrastructure for API management. The Kong API gateway is a widely used open source project, and Kong is a company built around supporting and building on top of the API gateway. Marco Palladino is the co-founder and CTO of Kong. He joins the show to tell the story of starting Kong eight years ago, and how the API gateway product evolved out of an API marketplace. Marco also discusses the architecture of Kong and his vision for how the product will develop in the future–including the Kong service mesh.

Originally published June 7, 2018 Moore’s Law states that the number of transistors in a dense integrated circuit doubles about every two years. Moore’s Law is less like a “law” and more like an observation or a prediction. Moore’s Law is ending. We can no longer fit an increasing amount of transistors in the same amount of space with a highly predictable rate. Dennard scaling is also coming to an end. Dennard scaling is the observation that as transistors get smaller, the power density stays constant. These changes in hardware trends have downstream effects for software engineers. Most importantly–power consumption becomes much more important. As a software engineer, how does power consumption affect you? It means that inefficient software will either run more slowly or cost more money relative to our expectations in the past. Whereas software engineers writing code 15 years ago could comfortably project that their code would get significantly cheaper to run over time due to hardware advances, the story is more complicated today. Why is Moore’s Law ending? And what kinds of predictable advances in technology can we still expect? John Hennessy is the chairman of Alphabet. In 2017, he won a Turing award (along with David Patterson) for his work on the RISC (Reduced Instruction Set Compiler) architecture. From 2000 to 2016, he was the president of Stanford University. John joins the show to explore the future of computing. While we may not have the predictable benefits of Moore’s Law and Dennard scaling, we now have machine learning. It is hard to plot the advances of machine learning on any one chart (as we explored in a recent episode with OpenAI). But we can say empirically that machine learning is working quite well in production. If machine learning offers us such strong advances in computing, how can we change our hardware design process to make machine learning more efficient? As machine learning training workloads eat up more resources in a data center, engineers are developing domain specific chips which are optimized for those machine learning workloads. The Tensor Processing Unit (TPU) from Google is one such example. John mentioned that chips could become even more specialized within the domain of machine learning. You could imagine a chip that is specifically designed for a LSTM machine learning model. There are other domains where we could see specialized chips–drones, self-driving cars, wearable computers. In this episode, John describes his perspective on the future of computing, and offers some framework for how engineers can adapt to that future.

Originally published November 4, 2016 Indie Hackers is a website that profiles independent developers who have made profitable software projects, usually without raising any money. These projects make anywhere from a few hundred dollars a month to more than $100,000 as in the case with park.io, one of the services profiled by Indie Hackers. Courtland Allen is the creator, engineer, and interviewer behind Indie Hackers. For each business that is profiled by Indie Hackers, Courtland conducts a short interview with the founder. Courtland joins the show to discuss the changing trends that are making it easier to bootstrap a software business if you are a capable developer–or even if you are a nontechnical person who understands how software works. Since Courtland and I are both in the business of interviewing engineers, we had a lot to talk about, and this is a fantastic episode.

New software abstractions always take advantage of the abstractions that have been built before. Software libraries allow us to import code that sits on the same host as a new program. Open source software let us copy and paste existing code, or clone entire repositories. Cloud providers offer hosted tools and APIs that we can leverage to develop scalable, easy-to-use infrastructure. When existing pieces of software are built into new software, the existing software becomes a dependency. Managing those dependencies is an engineering problem. Mya Pitzeruse is the founder of deps.cloud, a project with the goal of improving dependency changes across a company’s ecosystem. In today’s show, we talk about the modern dependency issues of a large company, and her perspective on how to address them. Mya has developed the project in public on FindCollabs, and we also spend some time talking about building in the open.

Web applications are used on a wide variety of platforms. On each of these platforms the web app needs to load properly and allow the user to navigate the website and interact with all of the user flows, such as sign-up, login, and the various read and write operations that make up the functionality of any website. It is difficult to ensure web application functionality across all platforms, because there are so many platforms. Different operating systems, different underlying hardware, different browsers, and different device form factors all create potential sources of suboptimal website functionality and performance. Testing web applications often involves the work of a manual quality assurance (QA) employee. The QA can simulate the procedures that a normal user would go through. This QA process ensures that the website is operating as expected. But the manual workflow can slow down software development. Gabriel-James Safar is a software engineer and the founder of Madumbo, which was acquired by Datadog. Madumbo was founded with the goal of making web application testing simpler by identifying errors in pages and enabling users to create test suites from recordings of user activity. This process simplifies and accelerates the testing process. Gabriel-James joins the show to talk about his experience building Madumbo, and his perspective on the modern application testing process. Full disclosure: Datadog is a sponsor of Software Engineering Daily.

HTTP is a protocol that allows browsers and web applications to communicate across the Internet. Everyone knows that HTTP is doing some important work, because “HTTP” is at the beginning of most URLs that you enter into your browser. You might be familiar with the request/response model, and HTTP request methods such as GET, PUT, and POST. But unless you have had a reason to learn more about the details of HTTP, you probably don’t know much more than that. Julia Evans is a software engineer and writer who creates Wizard Zines, a series of easy-to-read online magazines that explain technical software topics. Julia’s zines include “Linux Debugging Tools”, “Help! I Have A Manager!”, and recently “HTTP: Learn your browser’s language”. Her zines are a creative, innovative format for describing the world of software engineering while also exploring her own artistic pursuits in writing, design, and illustration. Julia was previously on the show to discuss Ruby profiling, and she returns to the show to discuss HTTP, as well as her creative process and goals with Wizard Zines.

Key Values is a platform where companies are profiled with descriptions of their company values. These profiles describe features such as work-life balance, company culture, daily routines, and strategy. Lynne Tye created Key Values with the goal of building a small business that would make money through connecting job seekers to companies with a culture that matched their own personal values system. Key Values has become highly successful, and Lynne is making enough money from the business to live comfortably. In a previous episode, Lynne and I discussed her founding story and the engineering of Key Values. Today’s episode picks up a few years later, with Lynne having found significant success with her own company. Lynne’s software business is an example of a growing trend: “Indie Hackers”. This trend was identified by Courtland Allen, founder of the Indie Hackers platform. Courtland is close friends with Lynne, and Lynne’s desire to start her own software company was influenced by her conversations with Courtland. At a certain point, Lynne was considering raising money and growing Key Values. She was accepted into Y-Combinator. But she decided to stick with the Indie Hackers route, and grow the business independently. Lynne joins the show to talk about the process of starting a software business, and the pivotal decisions she has made around financing, growth, and her own psychology.

The vision of code-free programming has existed for decades. Software engineers have always dreamed of empowering non-technical users with the same creative tools that programmers have access to. For many years, the underlying technology of the web was not powerful enough to make this dream a reality. Platforms such as WordPress, Squarespace, and Wix have allowed for some of the functionality of programming without writing code, but the scope of what those tools could accomplish was limited. Today, the web has caught up. Improvements in browser technology and client devices mean that the end user has access to more powerful technology. The API economy encapsulates large amounts of functionality into cheap, well-defined functionality. No-code platforms are finding success among developers and non-developers, after persisting for several years as their technology matured. Bubble is a code-free platform for building websites, startups, and internal tools. Emmanuel Straschnov and Joshua Haas are the founders of Bubble, and they join the show to tell the story of Bubble, and give their perspective on engineering, business, and the low-code movement. The story of Bubble is strikingly similar to that of no-code tools Airtable and Webflow, which we have covered in previous episodes. All of these products have taken years to get to maturity, with no shortcuts–only gritty, difficult engineering problems and performance improvements. Each of these no-code platforms has an inspiring story behind them, and persistent founders who eventually got their product to success.

“Serverless” is an execution model where applications are scheduled and deployed to servers that are not directly managed by the application developer. In serverless execution, an application only loads and operates when a user actually needs to get a response from that application. This saves on resources, because many applications do not need to run at all times–they only need to be available for user requests. The serverless model was popularized by Amazon Web Services Lambda. When Lambda first launched in 2015, it was an experimental product. Today, it is a widely used product and the market has validated the desire for serverless execution. Other cloud providers have introduced different models of serverless functionality including Google Cloud Functions, Azure Functions, and Fastly edge computing. Zack Bloom is director of product for product strategy at Cloudflare, and he joins the show to discuss Cloudflare’s model for serverless execution. Zack also discusses Cloudflare’s growing product line, including the fast, privacy-protecting DNS resolver 1.1.1.1. Zack is a rare mix of engineering, business strategy, and product vision, which made for a great conversation. For more content, you can check out our episodes about serverless technology and episodes about Cloudflare.

Data engineering is difficult. Companies want to be able to maximize the value they get from their large data sets, but there are so many steps required for deriving that value that most companies feel like they are always far behind the ideal state of where they could be. The cloud makes it cheap to save data. Tools like Spark and Snowflake give us usable APIs for simplifying the data processing. Workflow engines like Airflow help us visualize a complex execution path of a data pipeline. With all of this tooling, why is it so hard to make use of our data? Nick Schrock is the creator of Dagster, an open source system for building modern data applications. Nick is also the CEO of Elementl, a company that he is building around Dagster. Before creating Dagster, Nick worked at Facebook, where he co-created GraphQL. Nick returns to the show to discuss modern data engineering, and why it continues to be so difficult for engineers to be productive with their data.

Java programs run in a different environment than they did ten years ago. Modern infrastructure runs on containers sitting in a Kubernetes cluster. The optimal configuration for a Java program in that context is different than it was for an environment dominated by virtual machines and bare metal. When you are co-scheduling your services with each other, those services could be fighting for resources. You may want to optimize them with more ahead-of-time compilation. Quarkus is a system for accelerating Java performance through the use of GraalVM. In a previous show, we explored the basics of GraalVM. In today’s show, Guillaume Smet and Emmanuel Bernard join the show to describe an application of GraalVM: the acceleration of Java. Guillaume and Emmaneul are engineers at Red Hat, and are working on changes to the Java ecosystem that are informed by the cloud and the rise of Kubernetes.

Crash monitoring emerged as a software category over the last decade. Crash monitoring software allows developers to understand when their applications are crashing on client devices. For example, we have an app for Software Engineering Daily that people download on Android or iOS. Users download the app to their smartphone. When the user is playing an episode, and the app crashes, the details of the crash are sent to a server that collects all of these crash reports. Crash reports allow a company to understand where their application is breaking on client devices. This is important, since there are so many client surfaces to test, from iOS to Android to browsers. As a business, crash monitoring is a category that has some similarities to log management. There are lots of companies that offer crash monitoring. At first glance, it seems like a simple problem to solve. It seems like a market without winner-take-all or winner-take-most dynamics. But at scale, crash monitoring becomes a deeply complex engineering problem. From indexing to database choices to complex distributed systems tradeoffs, crash monitoring is not a simple business, and it promises to provide an extremely good business for the few companies who are able to out-execute the crowded market. James Smith is the CEO of Bugsnag, a company that makes crash monitoring and application stability tools. James returns to the show to discuss the growth-stage engineering challenges of error monitoring, and the business opportunities that come with them.

Software bugs cause unexpected problems at every company. Some problems are small. A website goes down in the middle of the night, and the outage triggers a phone call to an engineer who has to wake up and fix the problem. Other problems can be significantly larger. When a major problem occurs, it can cause millions of dollars in losses and requires hours of work to fix. When software unexpectedly breaks, it is called an incident. To triage these incidents, an engineer uses a combination of tools, including Slack, GitHub, cloud providers, and continuous deployment systems. These different tools emit updates that can be received by an incident response platform, which allow the on-call engineer to have the information they need centralized to more easily work through the incident. On-call rotation means that different people will be responsible for dealing with different incidents that occur. When an incident happens, the current engineer who is on-call may not be aware that a similar incident happened last week. It might be easier for the new engineer to triage the issue if they have insights about how the incident was managed during the first time. Chris Riley is a DevOps advocate with Splunk. He joins the show to discuss the application of machine learning to incident response. We discuss the different data points that are created during an incident, and how that data can be used to build models for different types of incidents, which can generate information to help the engineer respond appropriately to an incident. Full disclosure: Splunk is a sponsor of Software Engineering Daily.

Frontend software development has become as complex as backend development. There was a time when frontend web development was simple. There was a small number of JavaScript frameworks and templating systems. Your CSS was simple configuration for the colors on your webpage. Today, there is a giant ecosystem of frontend tools, APIs, and middleware delivering data to the user. Gatsby is a framework based on React that allows developers to build performant web applications. Gatsby is not easy to explain. In some ways, it is like a compiler for your website. Gatsby pulls in the data that you need to build your website, including CMS data, APIs, and markdown, and then links that information into React components and CSS. This happens on the server, so your user gets served a website that does not require lots of round trips as your website renders. Kyle Mathews and Sam Bhagwat are the founders of Gatsby, the company that is based around GatsbyJS. They join the show to describe their vision for the framework, and their vision for the company.

Drones will deliver food to us. Drones will be able to extinguish fires. Drones will be used to relay Internet signals and make the world more connected. These all sound like great ideas, so why aren’t there more drones in the sky today? There are many answers to that question, some of which relate to engineering and some of which are about the regulatory barriers. Chris Anderson is the CEO of 3D Robotics, a drone company which he started seven years ago. Before 3DR, Chris worked for many years as a journalist writing about technology and science. He was the editor-in-chief at Wired for 11 years, a writer for The Economist for 7 years, and spent 3 years at both of the leading scientific journals, Nature and Science. Chris also wrote The Long Tail, the influential 2004 book which described a set of emergent Internet trends. Chris joins the show for a discussion about drones, journalism, and his perspective on modern technology.

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets. In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications. Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively. We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

Every company has a software supply chain. A company builds its products from custom code, paid APIs, paid proprietary binaries, and open source software libraries. As the types of software available have increased, the management of the software supply chain has become complex. Large software companies have always needed to ensure the security of their software. With the growing variety of open source licenses, these companies also have to deal with an increased set of legal complexities. If an open source project is used in a way that violates an open source license, the company is subject to legal risk. FOSSA is a company that focuses on automating the management of open source compliance and security. Kevin Wang is the CEO of FOSSA, and he joins the show to discuss the modern issues of software licensing and his experience building a company.

Machine learning is widely understood by the software community. But it is still hard to build a company around machine learning, because there is not easy access to large, unique data sets. Scale is a platform for training and validating data that is used for machine learning. Most machine learning models are built with supervised learning. Labeled examples are analyzed to understand the mathematical correlations between those labels. The more labeled training examples there are, the more accurate the correlations will be. Today, we have high quality frameworks for writing the models. We have cheap cloud computing for training and deploying the models. The biggest factor that is preventing a wide variety of potential machine learning applications from existing is lack of access to large, labeled data sets. Scale gives developers an API for labeling images, sound, natural language, and video. Scale is used by self-driving car companies, Airbnb, OpenAI, retailers, and robotics companies. The product is used broadly and at high volume. Scale was started only three years ago, and recently raised $100m at a valuation above $1b, making it one of the fastest growing software companies in history. Alexandr Wang joins the show to discuss how Scale works, the future of machine learning, and the future of work. He also describes the complexities of building Scale, and how he manages his own psychological state.

Photo credit: Elisabeth Fall Ben Horowitz started Loudcloud with Marc Andreessen in 1999. He ran the company for eight years and chronicled his experience in his first book The Hard Thing About Hard Things. In his time running Loudcloud, the dotcom bubble burst, but Loudcloud needed cash so badly that he took the company public in 2001. Loudcloud went through layoffs, downsizing, and a difficult strategic maneuver in which Loudcloud sold its cloud provider business for cash, then used the core competency it had developed to create new software for building and running cloud services. This new software was the core product of the company Opsware, which was sold to HP in 2007 for $1.6bn. The Loudcloud story looks like a rational, straightforward execution in retrospect, but at many points in the timeline, Ben was unsure he was making the correct decision. As the subtitle of his first book states “there are no easy answers.” The Hard Thing About Hard Things tells the story of Loudcloud and Opsware in harrowing detail. Most founders of software companies will end up reading the book at some point when they are building their company, because there are so few books which capture the granular details of what it feels like to run a company. A CEO is completely alone in their understanding of the company. Nobody else has nearly as much information as the CEO–not the board, not the market, and not the employees. When you are a CEO, there is simply nobody to turn to who can give you the actionable advice that you wish you could have access to. And because there is nobody else, it means that the CEO’s own psychological state is extremely important. The Hard Thing About Hard Things provides a CEO with solace: while the CEO is alone within their company, they are not alone in the world. Every CEO has a set of issues which they have never faced before, and the CEO can learn to face those issues confidently and competently. Like any influential book, The Hard Thing About Hard Things presents the reader with useful answers, but also raises many questions. How can a normal person foster the mentality of a leader? How can a leader convince smart people to follow their direction? How can a seemingly crazy direction be framed as completely rational? The second book by Ben Horowitz is called What You Do Is Who You Are. This book surveys a set of case studies in leadership, including a Haitian slave revolt, the Mongol empire, and a dominant prison gang. By studying violent environments, Ben frames leadership in the context of the highest stakes. These stories are about life and death. When a leader’s performance is measured in blood, it frames the true nature of leadership in the starkest resolution. Ben uses each distilled example as a base case which inducts into broader applications: the cultures of Netflix, Facebook, Uber, and McDonalds are explored alongside editorials about Hillary Clinton and hip hop culture. Throughout all of these stories, the most important thread is continually reinforced: the leader creates the culture. The culture is the leader. What you do is who you are. Ben joins the show to discuss his writing, and how he has applied these beliefs to Andreessen Horowitz, the venture capital firm he co-founded and leads today.

Facebook leadership has a significant amount of engineers in its ranks, and engineers understand how to create an environment that appeals to other engineers. Engineers do not like working on projects that they are not interested in, so Facebook optimizes for matching engineers to enjoyable work. Engineers do not like taking orders from managers, so Facebook optimizes for a collaborative relationship between engineering and management. Arturo Bejar was a director of engineering at Facebook for more than five years. Arturo managed a team that built site integrity tools, as well as the Product Infrastructure team, which created the groundbreaking React and GraphQL open source tools. Arturo has a deep understanding of what engineers want from a manager, and this is apparent in the results of the teams he has worked with. Arturo also cares deeply about the human side of technology. When he was at Facebook, he started the Compassion Project, which looked at ways that technology could help support people through difficult times, and alleviate suicide and bullying. Arturo joins the show for a conversation about engineering management, humanity, and the ethos of Facebook. Articles by Arturo Bejar Managing Your Manager Challenges and Opportunities in Communication Online: Summary and Full Version

Chaos Monkeys is a book about Silicon Valley startups and Facebook. It is one of the most accurate books written about the modern technology industry, and captures both the negatives and the positives of software companies. Antonio Garcia Martinez is the author of Chaos Monkeys. He wrote the book after going through a gauntlet of prototypical Silicon Valley experiences. Antonio founded an adtech company called AdGrok. His company was funded by the startup accelerator Y-Combinator, and during his time at the company there were many dramatic events that Antonio tells in great detail. AdGrok was acquired by Twitter, and Antonio went to work at Facebook on its ads platform. Antonio is a fantastic writer. What makes Chaos Monkeys special is that it reads like a book written by an author who accidentally found himself in a technology career, rather than a technologist who opportunistically wrote a book. Note: This episode contains explicit language.

Facebook engineering is unique. Software is built at Facebook in a way that is distinctly different than any other company. In our series of shows about Facebook engineering, we have mostly covered the positive side of Facebook’s practices. In today’s show, we explore the downsides. Facebook moves fast. Engineers within the company must move fast, which can reduce the time spent on formal process and documentation. A fast pace can lead to a chaotic environment. In this kind of environment, not every employee thrives, and product development can sometimes suffer. But there are clear benefits to moving fast, and many software organizations could benefit from moving faster. Pete Hunt was an engineer with Facebook who worked on Facebook Video, Instagram, and React. Nick Schrock worked on core infrastructure at Facebook and was the co-creator of GraphQL. Pete and Nick helped create the Facebook culture, and are aware of the best and worst aspects of it. They join the show to reflect on their time at Facebook, and the downsides of moving fast.

Peter Deng has worked on most of Facebook’s major products: Newsfeed, Instagram, Oculus, and Messenger. These different products have different requirements, but are all part of the same ethos of connecting people through social networks. Facebook is a consumer product company that is powered by a strong engineering workforce. The relationship between product managers and engineering are two parts of a three-legged relationship: product, engineering, and design. Every major product within Facebook is built with the teamwork of product, engineering, and design. After almost ten years at Facebook, Peter joined Uber as the Head of Rider. At Uber, Peter works on a very different platform: a real-world, two-sided marketplace. Every change to the Uber platform has an impact on the economic relationship between riders and drivers. This creates a set of product development constraints that contrast with the social network of Facebook. Peter joins the show to share how he thinks about product management, and how the core competencies of a business inform product strategy.

Jeff Rothschild was one of the earliest engineers to join Facebook. In the 1990s, Jeff had co-founded Veritas Software and helped it to its IPO in 2004. After Veritas, Jeff worked on several other of his own companies. He was working with Accel Partners on investments when he started to learn about Facebook. Accel was an investor in Facebook, having made a large investment in 2005. After the investment, Jeff was helping the company with hiring. As Facebook built out its senior engineering leadership, Jeff realized that he would enjoy working with the company in a more active role. For the next ten years, Jeff Rothschild worked as a VP of infrastructure software at Facebook. He helped scale the company’s infrastructure and architect the technical strategy. By 2005, Jeff’s entrepreneurial background included software companies ranging from storage management to gaming. His diverse skill set was useful for setting product direction as well as solving novel engineering problems. Jeff joins the show to discuss the early days of Facebook and his philosophy of entrepreneurship.

In 2009, the Berkeley AMPLab was a center of innovation. Three projects from AMPLab have turned into successful open source projects and companies: Spark, Mesos, and Alluxio. Haoyuan Li was the creator of Alluxio, and he returns to the show to discuss his journey taking Alluxio from a research project to a company that has customers including Alibaba, Baidu, Wells Fargo, and Samsung. Alluxio is a distributed, fault-tolerant, tiered storage system. Alluxio allows application developers to think in terms of the latency that they require from their infrastructure rather than the details of different storage systems. Haoyuan discusses the process of integrating with gigantic companies like cloud providers, telecoms, and huge ecommerce companies. Alluxio is also hosting an upcoming conference, the Data Orchestration Summit November 7th at the Computer History Museum in Mountain View California. November 7, 2019 is the first open source Data Orchestration Summit in the Computer History Museum in Mountain View, CA. The summit brings together data engineers, data platform engineers, and data scientists to share their challenges and learnings from building and using modern analytics, AI, and cloud technologies. Featuring tech talks covering use cases, demos, best practices, and tutorials by industry experts from EA, Walmart, DBS Bank, Netflix, AWS, Rakuten, Tencent, Google, Baidu, Alibaba and more, with a focus on how to build cloud native analytics & AI platforms. Use code “SDE” to get a 50% discount.

Redis is an in-memory database that persists to disk. Redis is commonly used as an object cache for web applications. Applications are composed of caches and databases. A cache typically stores the data in memory, and a database typically stores the data on disk. Memory has significantly faster access times, but is more expensive and is volatile, meaning that if the computer that is holding that piece of data in memory goes offline, the data will be lost. When a user makes a request to load their personal information, the server will try to load that data from a cache. If the cache does not contain the user’s information, the server will go to the database to find that information. Alvin Richards is chief product officer with Redis Labs, and he joins the show to discuss how Redis works. We explore different design patterns for making Redis high availability, or using it as a volatile cache, and we talk through the read and write path for Redis data. Full disclosure: Redis Labs is a sponsor of Software Engineering Daily.

LinkedIn is a social network with petabytes of data. In order to store that data, LinkedIn distributes and replicates that data across a large cluster of machines running the Hadoop Distributed File System. In order to run calculations across its large data set, LinkedIn needs to split the computation up using MapReduce-style jobs. LinkedIn has been developing its data infrastructure since the early days of the Hadoop ecosystem. LinkedIn started using Hadoop in 2008, and in the last 11 years, the company has adopted streaming frameworks, distributed databases, and newer execution runtimes like Apache Spark. With the popularization of machine learning, there are more applications for data engineering than ever before. But the tooling around data engineering means that it is still hard for developers to find data sets, clean their data, and build reliable models. Carl Steinbach is an engineer at LinkedIn working on tools for data engineering. In today’s episode, Carl discusses the data platform inside LinkedIn, and the strategies that the company has developed around storing and computing large amounts of data. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

Cryptocurrencies are a fundamental computer science invention. Cryptocurrencies crashed in 2018 but the technology remains as promising as ever. Bitcoin is a decentralized currency, and a plausible end state that is implied by Bitcoin’s current trajectory is a permissionless, decentralized financial system. This idea of decentralized finance or “DeFi” begs numerous questions: who will build the companies that provide the infrastructure for decentralized finance? Who will be the lenders? Who will be the credit agencies? Who will be the escrow services? How big will the teams need to be? Will these systems be built on smart contracts, or can it be done with centralized cloud providers? Haseeb Qureshi is a managing partner with DragonFly Capital and a frequent guest on SE Daily. He returns to the show to discuss his thesis on what kinds of crypto companies make sense, and how he thinks about investments into crypto startups. Haseeb recently wrote an article about how to build a crypto startup.

Dark Lang is a programming language that is tightly integrated with the cloud. Dark takes an opinionated approach that most developers are going to want to run their applications in the cloud, and this perspective influences how Dark looks at deployments, IDEs, exception handling, and other aspects of software development. Paul Biggar is the founder of CircleCI, and ran the company for eight years before leaving to found Dark with Ellen Chisa. Ellen is a software engineer and the CEO of Dark. Paul and Ellen join the show to give their perspective on modern software engineering, and why it was time to build a new high level language that assumes the presence of a cloud. It is difficult to get programmers to adopt a new language. It is even harder to get those programmers to pay for products built around that language. But the timing could be perfect for Dark. Software development is undergoing tremendous change–and many of these changes work to Dark’s advantage, such as the growing adoption of feature flags, low code tools, and sophisticated continuous delivery workflows. Whether or not Dark is a success, it is a bold project, and the team is working on something they believe in. We also discussed AWS–and whether the largest cloud provider has an obligation to contribute back to the open source community.

Apache Kafka was created at LinkedIn. Kafka was open sourced in 2011, when the company was eight years old. By that time, LinkedIn had developed a social network with millions of users. LinkedIn’s engineering team was building a range of externally facing products and internal tools, and many of these tools required a high-throughput system for publishing data and subscribing to topics. Kafka was born out of this need. Over time, Kafka’s importance within LinkedIn has only grown. Kafka plays a central role for services, log management, data engineering, and compliance. LinkedIn might be the biggest user of Apache Kafka in the entire software industry. Kafka has many use cases, and it is likely that they are almost all on display within LinkedIn. Nacho Solis is a senior software engineering manager at LinkedIn, where he helps teams build infrastructure for Kafka, as well as Kafka itself. Nacho joins the show to discuss the history of Kafka at LinkedIn, and the challenges of managing such a large deployment of Kafka. We also talk about streaming, data infrastructure, and more general problems in the world of engineering management. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

Modern applications are distributed systems. These applications require an installation mechanism that can run and update the software across multiple nodes. When a SaaS company starts to work with large enterprise customers, that company needs to figure out a way to deliver their product to the enterprise. This requires the SaaS company to deploy the product to whatever infrastructure the enterprise is running. Some enterprises use on-prem infrastructure. Some use AWS. Some use a variety of cloud providers and on-premise servers. A SaaS company with limited resources must be able to have a standard deployment model that satisfies all of these different use cases. Ev Kontsevoy is the CEO of Gravitational, a company that builds software for application delivery. Ev’s company maintains Gravity, an open source tool for application imaging and delivery. Ev was also the founder of Mailgun, a popular email API service. Mailgun was acquired by Rackspace, and in his time running Mailgun, Ev became deeply aware of the problems faced by developers and operators who manage server infrastructure. Ev joins the show to discuss his experience building companies and the state of modern infrastructure. Full disclosure: Gravitational is a sponsor of Software Engineering Daily.

Databases go offline. Services fail to scale up. Deployment errors can cause an application backend to get DDoS’d. When an event happens that prevents your company from operating as expected, it is known as an incident. Software teams respond to an incident by issuing a fix. Sometimes that fix returns the software to its ideal state. Other times the software remains in a degraded state, and it takes more fixing to return the software to the place it should be. One way that a software team can learn from an incident is through incident reproduction. When an incident is turned into a reproducible system, it becomes a predictable training exercise rather than a surprising and painful outage. Tammy Butow is an engineer with Gremlin, a company that makes chaos engineering software. Chaos engineering is the process of creating controlled experiments that simulate outages. Tammy joins the show to discuss common incident types, and how those can be made reproducible for training exercises.

Diffbot is a knowledge graph that allows developers to interface with the unstructured web as if it was a structured database. In today’s show, Diffbot CEO Mike Tung returns for a second discussion about how he has built Diffbot and how Diffbot is used. The web has many different entities. Web pages, topics, people, stories, articles, companies, and much more. Humans use a search engine to find answers to their questions within web pages. Machines need to find answers to these kinds of questions as well, but a machine is not sophisticated enough to figure out answers from an unstructured web page. Diffbot brings structure to those webpages, and gives them an API interface for developers to build on top of. In order to create this system in a cost-efficient manner, Diffbot runs its own data centers, where web scraping, machine learning, and API infrastructure are all used to build the Diffbot application. Mike joins me for an interview about creating Diffbot, as well as his strategy for running the business.

Render is a cloud provider built on top of Amazon Web Services and Google Cloud. Render uses the compute abstractions provided by the major cloud providers to build a second layer cloud provider with the goal of providing a better user experience. Anurag Goel is the founder of Render, and he returns to the show to discuss how Render works, and why there is a need for a new cloud provider. Everyone knows that the market for cloud providers is gigantic, so why are so few companies pursuing it? From Anurag’s perspective, there is no good reason. AWS, Google, Azure, Digital Ocean, and Heroku have only explored a small percentage of the potential ways a cloud provider could be built. Anurag shares his strategy for building Render, and also talks through his belief around modern software engineering, including his belief that developers mostly choose their tools based on what they read from popular websites rather than what solves their problems–a phenomenon which he describes as “fashion driven development.”

Webflow is a platform for building applications without programming. Software engineering has barely been around for 30 years. Over that period of time, there have been many attempts to create a platform that allows for the creation of software without writing a line of code. Most of these systems have not been able to fulfill that task. And this should come as no surprise. It is hard enough to build an application if you know how to program. Vlad Magdolin has been working on Webflow for more than seven years. He has persisted through multiple failed attempts at building Webflow, and pushed past continuous rejection from investors who did not see the viability of his vision. As Vlad patiently worked on Webflow with his two co-founders, the power of the web browser slowly improved. V8 became a powerful runtime that could deliver the performance necessary to build applications visually in the browser. The unmet goals of past WYSIWYG application platforms faded into irrelevance, as Webflow came into being and allowed for an entirely new type of software development, driven by a visual interface in the browser. Webflow is one of the coolest, most ambitious software platforms in existence. Vlad joins the show to discuss Webflow and the future of software development.

Data visualization is the presentation of data in a way that emphasizes certain qualities about that data. Data visualization can be used to prove a specific point, or it can be used as a depiction of a data set to be explored. Data visualization is used in consumer software products as well as backend engineering systems such as logging data. As tools for data visualization have improved, data applications can consume more data at a faster pace. Browsers and mobile phones have improved, giving us the power to render high fidelity, complex visualizations in near-real time. Sherman Wood and Chad Lumley are engineers working on Jaspersoft, an embedded analytics tool from TIBCO. Embedded analytics is a type of software that allows for the creation of data visualizations inside of an application. Sherman and Chad join the show to discuss techniques for data visualization and how the field has evolved. Full disclosure: TIBCO is a sponsor of Software Engineering Daily.

The ability to create software is a superpower. But software education is not evenly distributed. Ryan Carson started Treehouse to provide a high quality education system for anyone to learn how to build software. On a previous episode, Ryan and I discussed the field of programming education. Ryan returns to the show for a conversation about building Treehouse, and the company’s expansion from an online programming school to a platform for technology apprenticeship. Thousands of people learn to program on Treehouse. Ryan’s goal is to connect those new programmers to companies where they can learn to apply those software skills to working at a technology company. Treehouse’s apprenticeship program is used by Airbnb, Adobe, Mailchimp, and others. By combining an online education platform with an apprenticeship program, Treehouse has created a business model that could allow it to grow significantly bigger even in an atmosphere where learning to program has been commodified to a large extent. The economic model of Treehouse contrasts with in-person bootcamps such as Hack Reactor and online income sharing platforms such as Lambda School. It was a great conversation about modern technology, the future of education, and the strategy of building a successful business.

Video surveillance impacts human lives every day. On most days, we do not feel the impact of video surveillance. But the effects of video surveillance have tremendous potential. It can be used to solve crimes and find missing children. It can be used to intimidate journalists and empower dictators. Like any piece of technology, video surveillance can be used for good or evil. Video recognition lets us make better use of video feeds. A stream of raw video doesn’t provide much utility if we can’t easily model its contents. Without video recognition, we must have a human sitting in front of the video to manually understand what is going on in that video. Veronica Yurchuk and Kosh Shysh are the founders of Traces.ai, a company building video recognition technology focused on safety, anonymity, and positive usage. They join the show to discuss the field of video analysis, and their vision for how video will shape our lives in the future.

Data privacy policies have changed how software organizations need to operate. As consumer preferences have shifted in favor of strong privacy, software companies are having to examine their policies around data collection and retention. Many software companies were started in a time with different norms around data. Building a new application that is compliant with GDPR is hard. Updating an existing application to align with GDPR is even harder. Joshua Prismon is chief architect at FICO, a company that builds systems around credit scoring. Joshua joins the show to discuss how a large company like FICO has responded to changing consumer preferences through changes in its software architecture and engineering.

Indie Hackers is a platform for independent software businesses to discuss strategy and find inspiration. Courtland Allen founded Indie Hackers with the goal of sharing the stories of these businesses, and the company has become a thriving community of entrepreneurs, engineers, and creators. Business is a creative medium. The definition of a successful business is as subjective as the rules for what makes a successful work of art. A business owner can be miserable running a company that generates millions of dollars a year, and a new entrepreneur can feel ecstatic from making their first $5 sale. Indie Hackers is a platform that is impossible to define in relation to things you have seen before. It is a media company with a podcast that most SE Daily listeners will probably enjoy. It is a social platform for learning how modern software companies are built. And it is a place where makers post their own progress on their creative projects (I have posted mine here). Courtland was on the show three years ago to discuss the Indie Hackers movement in its nascent stages. Courtland returns to the show to discuss the thriving platform as it exists today, and a wide ranging conversation about software, game theory, and podcasting.

Scott Belsky founded Behance in 2006. Behance is a social platform where designers and creators share their work. Scott was motivated to start Behance due to his desire to combine his love for creativity with his desire to create a business. After 6 years of work, Behance was acquired by Adobe for more than $150 million. Today, Scott works as the Chief Product Officer at Adobe. Behance’s journey from idea to acquisition is told by Scott in his book The Messy Middle. His book chronicles the difficult, winding journey that an entrepreneur must take in order to succeed, and contains some harrowing stories. Scott has a gritty personality, which was required to endure the ups and downs of Behance. Scott joins the show to discuss the story of Behance, and the lessons of his life as an entrepreneur.

Large companies have multiple databases, multiple data formats, and multiple applications that need to use the data. Every data engineer needs to move data between these different components of a system. Moving data between different parts of a system is often called “ETL”, an acronym for “Extract, Transform, Load.” Data engineers spend much of their time writing code for ETL. This ETL code often moves data from a source such as a distributed file system into a data warehouse such as Amazon Redshift. ETL code also is used to take data out of platforms such as Google Analytics, and put that data into a system where it can be joined with internal data sets. George Fraser is the CEO of Fivetran, a company that builds data connectors to improve the data engineering process. George joins the show to discuss modern data warehousing, and the business of building Fivetran. Data connectors may sound like a simple or trivial product, but it turns out to be a gigantic opportunity.

The development of self-driving cars is one of the biggest technological changes that is under way. Across the world, thousands of engineers are working on developing self-driving cars. Although it still seems far away, self-driving cars are starting to feel like an inevitability. This is especially true if you spend much time in downtown San Francisco, where you will see a self-driving car being tested every day. Much of the time, that self-driving car will be operated by Cruise. Cruise is a company that is building a self-driving car service. The company has hundreds of engineers working across the stack, from computer vision algorithms to automotive hardware. Cruise’s engineering requires engineers who can work with cloud tools as well as low-latency devices. It also requires product developers and managers to lead these different teams. The field of self-driving is very new. There is not much literature available on how to build a self-driving car. There is even less literature on how to manage a team of engineers that are building, testing, and deploying software and hardware for real cars that are driving around the streets of San Francisco. Mo Elshenawy is VP of engineering at Cruise, and he joins the show to talk about the engineering that is required to develop fully self-driving car technology, as well as how to structure teams to align the roles of product design, software engineering, testing, machine learning, and hardware. Full disclosure: Cruise is a sponsor of Software Engineering Daily.

Investors often use the term “moat” to describe the durable competitive advantage of a company. When an investor puts money into a company, they are making that investment based on a valuation. That valuation is subjective–it is how much the investor thinks the company is worth. A valuation is determined by the present value of future cash flows. What are the future cash flows of a company? In order to figure that out, the investor needs to know how the business will look in the future. This is why moats are so important. If an investor looks at how much money a business made this year, it does not tell the investor very much information about how much money the business will make in the future. If the business has a durable competitive advantage, then that means that the cash flows of the company are also durable. It also means that any compounding of those cash flows will be durable growth. It is easy to understand durability for some businesses. Why do we keep using Google? Because there is no substitute search engine that is so integrated with our daily lives. Why do we keep using Facebook? Because there is no other social networking company that has all of our friends and family. But what about companies with substitutes? There are lots of cloud providers, log management companies, and analytics providers. These are crowded markets, and yet in each of the crowded markets there seems to be a dominant player who captures the most market share. Why is that? How do software companies in competitive markets develop a moat? Astasia Myers is a venture investor with Redpoint, a software investment firm that makes large bets on technology companies. Astasia joins the show to discuss how software companies form competitive advantages, as well as several specific markets such as log management and cloud cost optimization.

Software engineering is a new field. There are theories about how we should be building our systems, but these theories might change over time. The same is true for engineering management. There are many successful examples of companies scaling with the management hierarchies pioneered by Microsoft and Google, but since everyone knows that those techniques work, they get continually copied. Of course, there is tremendous risk in pioneering a brand new management structure. There have been examples of successful software companies that have suffered tremendously due to a trial of an exciting new management structure. New software is much easier and safer to try than revolutionary new management structures. New software is easier to try in a company with a strong engineering team, because that team is equipped to assess the new software and figure out if it is actually solving a problem that the company has. Uma Chingunde is an engineering manager at Stripe on the compute team. Uma has worked in management for a decade, and has worked in virtualization in infrastructure for even longer than that. Uma joins the show to give her perspective on management of engineers as well as management of compute infrastructure. We discussed some timeless principles of engineering management, as well as contemporary ideas around virtualization and compute.

Ethos Life Insurance is a software company that sells life insurance products. Software is reshaping established industries such as banking, insurance, and manufacturing. In these large, established industries, incumbents are adopting new technology as fast as they can, but the new technology needs to be integrated with the old technology. The slow rate of technology adoption by incumbents creates an opportunity for new companies to spring up who are building their entire company from scratch, with updated software. Insurance is a gigantic market that is dominated by companies which have been around for 50-100 years. The established players in the insurance industry are trusted brands, but many of them have not significantly updated their technology from legacy techniques of pricing risk. Vipul Sharma is the VP of engineering and Lingke Wang is the co-founder of Ethos Life Insurance, and they join the show to describe the insurance business, the technical problems, and the software stack of modern insurance company. Ethos has more than fifty employees and is growing rapidly, so it is a great case study in scaling a modern company in an established market.

Isolation is a fundamental concept in computer science. Software workloads are isolated from each other in order to keep resource access cleanly separated. When programs are properly isolated, it is easier for the programmer to reason about the memory safety of that program. When a program is not properly isolated, it can lead problems such as security flaws where one program can access the information that should be exclusive to a different program. Poor isolation can also lead to garbage collection problems, or running out of disk space. Isolation takes many forms, including individual processes, containers, and virtual machines. The techniques for isolation evolve over time. A more recent technology that can assist with isolation is WebAssembly, a newer execution system that can run a variety of languages that compile down into the WebAssembly binary format. For previous episodes about WebAssembly, you can listen to some of the shows in our archives. Tyler McMullen is the CTO at Fastly, a cloud provider that focuses on edge computing systems such as content delivery networking. Tyler has written and spoken about WebAssembly in detail. He joins the show to talk about computational isolation, and how WebAssembly presents new efficiencies for engineers looking to isolate their workloads. Full disclosure: Fastly, where Tyler works, is a sponsor of Software Engineering Daily.

Cloud Foundry is a system for managing distributed applications. Cloud Foundry was released in 2011, and has been widely adopted by enterprises that need a platform for deploying and scaling the applications that run within their company. The ecosystem around Cloud Foundry includes systems for continuous delivery, pubsub messaging, and containerization. Abby Kearns is the executive director at Cloud Foundry Foundation, a nonprofit with the goal of raising awareness and adoption of the Cloud Foundry open source project. Abby joins the show to discuss her work with Cloud Foundry, and how the ecosystem has evolved with the maturity of distributed computing. We also talk about what enterprises need from an application runtime platform and how Cloud Foundry has adopted Kubernetes.