Software Engineering Daily

The ability to create software is a superpower. But software education is not evenly distributed. Ryan Carson started Treehouse to provide a high quality education system for anyone to learn how to build software. On a previous episode, Ryan and I discussed the field of programming education. Ryan returns to the show for a conversation about building Treehouse, and the company’s expansion from an online programming school to a platform for technology apprenticeship. Thousands of people learn to program on Treehouse. Ryan’s goal is to connect those new programmers to companies where they can learn to apply those software skills to working at a technology company. Treehouse’s apprenticeship program is used by Airbnb, Adobe, Mailchimp, and others. By combining an online education platform with an apprenticeship program, Treehouse has created a business model that could allow it to grow significantly bigger even in an atmosphere where learning to program has been commodified to a large extent. The economic model of Treehouse contrasts with in-person bootcamps such as Hack Reactor and online income sharing platforms such as Lambda School. It was a great conversation about modern technology, the future of education, and the strategy of building a successful business.

Video surveillance impacts human lives every day. On most days, we do not feel the impact of video surveillance. But the effects of video surveillance have tremendous potential. It can be used to solve crimes and find missing children. It can be used to intimidate journalists and empower dictators. Like any piece of technology, video surveillance can be used for good or evil. Video recognition lets us make better use of video feeds. A stream of raw video doesn’t provide much utility if we can’t easily model its contents. Without video recognition, we must have a human sitting in front of the video to manually understand what is going on in that video. Veronica Yurchuk and Kosh Shysh are the founders of Traces.ai, a company building video recognition technology focused on safety, anonymity, and positive usage. They join the show to discuss the field of video analysis, and their vision for how video will shape our lives in the future.

Data privacy policies have changed how software organizations need to operate. As consumer preferences have shifted in favor of strong privacy, software companies are having to examine their policies around data collection and retention. Many software companies were started in a time with different norms around data. Building a new application that is compliant with GDPR is hard. Updating an existing application to align with GDPR is even harder. Joshua Prismon is chief architect at FICO, a company that builds systems around credit scoring. Joshua joins the show to discuss how a large company like FICO has responded to changing consumer preferences through changes in its software architecture and engineering.

Indie Hackers is a platform for independent software businesses to discuss strategy and find inspiration. Courtland Allen founded Indie Hackers with the goal of sharing the stories of these businesses, and the company has become a thriving community of entrepreneurs, engineers, and creators. Business is a creative medium. The definition of a successful business is as subjective as the rules for what makes a successful work of art. A business owner can be miserable running a company that generates millions of dollars a year, and a new entrepreneur can feel ecstatic from making their first $5 sale. Indie Hackers is a platform that is impossible to define in relation to things you have seen before. It is a media company with a podcast that most SE Daily listeners will probably enjoy. It is a social platform for learning how modern software companies are built. And it is a place where makers post their own progress on their creative projects (I have posted mine here). Courtland was on the show three years ago to discuss the Indie Hackers movement in its nascent stages. Courtland returns to the show to discuss the thriving platform as it exists today, and a wide ranging conversation about software, game theory, and podcasting.

Scott Belsky founded Behance in 2006. Behance is a social platform where designers and creators share their work. Scott was motivated to start Behance due to his desire to combine his love for creativity with his desire to create a business. After 6 years of work, Behance was acquired by Adobe for more than $150 million. Today, Scott works as the Chief Product Officer at Adobe. Behance’s journey from idea to acquisition is told by Scott in his book The Messy Middle. His book chronicles the difficult, winding journey that an entrepreneur must take in order to succeed, and contains some harrowing stories. Scott has a gritty personality, which was required to endure the ups and downs of Behance. Scott joins the show to discuss the story of Behance, and the lessons of his life as an entrepreneur.

Large companies have multiple databases, multiple data formats, and multiple applications that need to use the data. Every data engineer needs to move data between these different components of a system. Moving data between different parts of a system is often called “ETL”, an acronym for “Extract, Transform, Load.” Data engineers spend much of their time writing code for ETL. This ETL code often moves data from a source such as a distributed file system into a data warehouse such as Amazon Redshift. ETL code also is used to take data out of platforms such as Google Analytics, and put that data into a system where it can be joined with internal data sets. George Fraser is the CEO of Fivetran, a company that builds data connectors to improve the data engineering process. George joins the show to discuss modern data warehousing, and the business of building Fivetran. Data connectors may sound like a simple or trivial product, but it turns out to be a gigantic opportunity.

The development of self-driving cars is one of the biggest technological changes that is under way. Across the world, thousands of engineers are working on developing self-driving cars. Although it still seems far away, self-driving cars are starting to feel like an inevitability. This is especially true if you spend much time in downtown San Francisco, where you will see a self-driving car being tested every day. Much of the time, that self-driving car will be operated by Cruise. Cruise is a company that is building a self-driving car service. The company has hundreds of engineers working across the stack, from computer vision algorithms to automotive hardware. Cruise’s engineering requires engineers who can work with cloud tools as well as low-latency devices. It also requires product developers and managers to lead these different teams. The field of self-driving is very new. There is not much literature available on how to build a self-driving car. There is even less literature on how to manage a team of engineers that are building, testing, and deploying software and hardware for real cars that are driving around the streets of San Francisco. Mo Elshenawy is VP of engineering at Cruise, and he joins the show to talk about the engineering that is required to develop fully self-driving car technology, as well as how to structure teams to align the roles of product design, software engineering, testing, machine learning, and hardware. Full disclosure: Cruise is a sponsor of Software Engineering Daily.

Investors often use the term “moat” to describe the durable competitive advantage of a company. When an investor puts money into a company, they are making that investment based on a valuation. That valuation is subjective–it is how much the investor thinks the company is worth. A valuation is determined by the present value of future cash flows. What are the future cash flows of a company? In order to figure that out, the investor needs to know how the business will look in the future. This is why moats are so important. If an investor looks at how much money a business made this year, it does not tell the investor very much information about how much money the business will make in the future. If the business has a durable competitive advantage, then that means that the cash flows of the company are also durable. It also means that any compounding of those cash flows will be durable growth. It is easy to understand durability for some businesses. Why do we keep using Google? Because there is no substitute search engine that is so integrated with our daily lives. Why do we keep using Facebook? Because there is no other social networking company that has all of our friends and family. But what about companies with substitutes? There are lots of cloud providers, log management companies, and analytics providers. These are crowded markets, and yet in each of the crowded markets there seems to be a dominant player who captures the most market share. Why is that? How do software companies in competitive markets develop a moat? Astasia Myers is a venture investor with Redpoint, a software investment firm that makes large bets on technology companies. Astasia joins the show to discuss how software companies form competitive advantages, as well as several specific markets such as log management and cloud cost optimization.

Software engineering is a new field. There are theories about how we should be building our systems, but these theories might change over time. The same is true for engineering management. There are many successful examples of companies scaling with the management hierarchies pioneered by Microsoft and Google, but since everyone knows that those techniques work, they get continually copied. Of course, there is tremendous risk in pioneering a brand new management structure. There have been examples of successful software companies that have suffered tremendously due to a trial of an exciting new management structure. New software is much easier and safer to try than revolutionary new management structures. New software is easier to try in a company with a strong engineering team, because that team is equipped to assess the new software and figure out if it is actually solving a problem that the company has. Uma Chingunde is an engineering manager at Stripe on the compute team. Uma has worked in management for a decade, and has worked in virtualization in infrastructure for even longer than that. Uma joins the show to give her perspective on management of engineers as well as management of compute infrastructure. We discussed some timeless principles of engineering management, as well as contemporary ideas around virtualization and compute.

Ethos Life Insurance is a software company that sells life insurance products. Software is reshaping established industries such as banking, insurance, and manufacturing. In these large, established industries, incumbents are adopting new technology as fast as they can, but the new technology needs to be integrated with the old technology. The slow rate of technology adoption by incumbents creates an opportunity for new companies to spring up who are building their entire company from scratch, with updated software. Insurance is a gigantic market that is dominated by companies which have been around for 50-100 years. The established players in the insurance industry are trusted brands, but many of them have not significantly updated their technology from legacy techniques of pricing risk. Vipul Sharma is the VP of engineering and Lingke Wang is the co-founder of Ethos Life Insurance, and they join the show to describe the insurance business, the technical problems, and the software stack of modern insurance company. Ethos has more than fifty employees and is growing rapidly, so it is a great case study in scaling a modern company in an established market.

Isolation is a fundamental concept in computer science. Software workloads are isolated from each other in order to keep resource access cleanly separated. When programs are properly isolated, it is easier for the programmer to reason about the memory safety of that program. When a program is not properly isolated, it can lead problems such as security flaws where one program can access the information that should be exclusive to a different program. Poor isolation can also lead to garbage collection problems, or running out of disk space. Isolation takes many forms, including individual processes, containers, and virtual machines. The techniques for isolation evolve over time. A more recent technology that can assist with isolation is WebAssembly, a newer execution system that can run a variety of languages that compile down into the WebAssembly binary format. For previous episodes about WebAssembly, you can listen to some of the shows in our archives. Tyler McMullen is the CTO at Fastly, a cloud provider that focuses on edge computing systems such as content delivery networking. Tyler has written and spoken about WebAssembly in detail. He joins the show to talk about computational isolation, and how WebAssembly presents new efficiencies for engineers looking to isolate their workloads. Full disclosure: Fastly, where Tyler works, is a sponsor of Software Engineering Daily.

Cloud Foundry is a system for managing distributed applications. Cloud Foundry was released in 2011, and has been widely adopted by enterprises that need a platform for deploying and scaling the applications that run within their company. The ecosystem around Cloud Foundry includes systems for continuous delivery, pubsub messaging, and containerization. Abby Kearns is the executive director at Cloud Foundry Foundation, a nonprofit with the goal of raising awareness and adoption of the Cloud Foundry open source project. Abby joins the show to discuss her work with Cloud Foundry, and how the ecosystem has evolved with the maturity of distributed computing. We also talk about what enterprises need from an application runtime platform and how Cloud Foundry has adopted Kubernetes.

A new software product usually starts with a single database. That database manages the tables for user accounts and basic transactions. When a product becomes popular, the database grows in size. There are more transactions and more users. A company grows around that product, and the company starts to accumulate more data in different sources. Analytics systems, time series databases, and logging tools start to generate data. Moving this data around between systems starts to become complicated. Apache Kafka is often used as a system for moving data between these different systems, performing transformations, and generating aggregations and summaries of these large quantities of data. Robin Moffatt works at Confluent, and has written numerous articles about how to move data between systems and design effective workflows for data pipelines. Robin joins the show to talk about modern data platforms and databases, and the patterns for using Kafka to connect those systems to each other. If you are interested in learning more about how companies are using Kafka, the Kafka Summit in San Francisco is September 30th – October 1st. Companies like LinkedIn, Uber, and Netflix will be talking about how they use Kafka. Full disclosure: Confluent (the company where Tim works) is a sponsor of Software Engineering Daily.

A software company needs to get many things right in order to be successful. Having a useful product with solid engineering is only the beginning. ReadMe was started five years ago. The company solved a seemingly simple problem: documentation for software products. If you have worked as a software engineer, you have looked at documentation. You know that there is a wide range of quality among the documentation that exists for different software products. ReadMe solved the problem of documentation-as-a-service, and it solved the problem better than any other company in the market. But after building a great business around documentation, the direction in which to take the business was unclear. How do you expand a documentation software business? Should you start building API management systems? Should you build lead generation tools? Should you try to sell products to developer evangelists? Greg Koberger is the CEO of ReadMe, and he joins the show to talk about the business, the strategy, and the fundraising process for ReadMe. Greg was previously on the show very early on, and he stood out as a guest that was remarkably friendly and willing to talk about a wide range of topics. In the previous show with Greg, he was one year into his business. Today, it has been five years since ReadMe was started, and we catch up on how his business has evolved since our first interview. ReadMe had become a profitable business very early in its life. In Silicon Valley, a quickly profitable business is the exception. Rather than structuring the finances of ReadMe with the expectation of successive funding rounds every 18-24 months, Greg took a slower approach. His company grew at a rate that was affordable while maintaining that profitability. While ReadMe maintained profitability from its core product of documentation software, Greg patiently thought about what product to build next within the company. A software company will typically expand into adjacent markets, or offer products that their current users are willing to pay additional money for. Eventually, ReadMe found its second product: developer metrics. Greg and his team figured out that gathering metrics around how APIs are being consumed has synergies with the core ReadMe product. With a second product, ReadMe was now able to forecast significant growth. The company also gained an incentive to raise money. With additional money, ReadMe would be able to deepen the developer metrics product, hire a bigger team, and think much bigger than they would have been able to as a single-product company with a smaller total addressable market. ReadMe raised a series A from Accel, one of the most respected venture capital firms in the business, and the company is now entering a new period in its development. ReadMe is a fascinating case study in balancing ambition with financial discipline at a software company.

Open source software is very new. Open source has existed for less than 30-40 years, depending on who you ask. The idea of open source was popularized by Linux, and open source software started to get heavily commercialized in the 1990s. By the early 2000s, open source was used by nearly every large software company. In recent years, most of the new databases, web frameworks, user interface libraries, and other software primitives are being built in the open. Dirk Hohndel is VP and chief open source officer at VMware, where he works on strategies around Linux, containers, Kubernetes, and other open source software. Dirk joins the show to talk about his long history with open source and the current state of the open source ecosystem.

Modern databases consist of multiple servers that host the data in a distributed fashion. Using multiple servers allows a database to be resilient to the failure of any one database node, because copies of the data are shared to other servers. A multi-node setup also lets the database grow beyond the size of data that could be hosted on a single node. Although a distributed database gains in scalability and resiliency, a database that runs across multiple nodes has a variety of problems that are not faced by a database running on a single node. Every operation with a distributed database becomes more complex than the single-node database. For example, if you make a query to your distributed database, you might not be able to rely on the answer that you get from a single database node, because the other database nodes might have been involved in transactions that have not propagated to all of the nodes. Aly Cabral is a lead product manager at MongoDB and co-author of a paper on causal consistency in MongoDB. Aly joins the show to discuss the engineering of distributed databases and her experience architecting MongoDB. Full disclosure: MongoDB is a sponsor of Software Engineering Daily.

Ever since Apache Kafka was open sourced from LinkedIn, it has been used to solve a wide variety of problems in distributed systems and data engineering. Kafka is a distributed messaging queue that is used by developers to publish messages and subscribe to topics with a certain message type. Kafka allows information to flow throughout a company such that multiple systems can consume the messages from a single sender. In previous shows, we have covered design patterns within Kafka, Kafka streams, event sourcing with Kafka, and many other subjects relating to the technology. Kafka is broadly useful, and new strategies for using Kafka continue to emerge as the open source project develops new functionality and becomes a platform for data applications. In today’s episode, Tim Berglund returns to Software Engineering Daily for a discussion of how applications are built today using Kafka–including systems that are undergoing a refactoring, data engineering applications, and systems with a large number of communicating services. If you are interested in learning more about how companies are using Kafka, the Kafka Summit in San Francisco is September 30th – October 1st. Companies like LinkedIn, Uber, and Netflix will be talking about how they use Kafka. Full disclosure: Confluent (the company where Tim works) is a sponsor of Software Engineering Daily.

A new employee at a software company needs access to a variety of tools. In order to get started working, the employee might need Slack, email, Google Docs, and Amazon Web Services, and all of these require an account with a username and password. Setting up all of these accounts can be time consuming, because the company needs to go into their admin portal and create the accounts. The accounts need to have the right security policies and configuration settings. And when the employee leaves the company, all of these accounts need to be shut down. Okta is a company that builds identity and access management software, such as an “SSO (single-sign on)” tool that allows users to log into all of these different types of accounts using only an Okta login. Okta was started in 2009 and has grown steadily since then, going public in 2017. Hector Aguilar is the president of technology at Okta and he joins the show to talk about the software stack of Okta and how the company has evolved over time as it has become a core infrastructure provider and hired a large engineering team.

Amazon Web Services first came out in 2006. It took several years before the software industry realized that cloud computing was a transformative piece of technology. Initially, the common perspective around cloud computing was that it was a useful tool for startups, but would not be a smart option for large, established businesses. Cloud computing was not considered economical nor secure. Today, that has changed. Every company that writes software is figuring out how to utilize the cloud. Software companies with on-prem servers are migrating old applications to the cloud, and most companies that have started in the last decade do not even have physical servers. Applications that are started on the cloud are referred to as “cloud-native.” The architecture of cloud-native applications is a newer topic of discussion, and some software patterns that became established in the pre-cloud era make less sense today. Cornelia Davis is VP of technology at Pivotal and the author of Cloud Native Patterns, a book about developing applications in the distributed, virtual world of the cloud. Cornelia was previously on the show to discuss Cloud Foundry. In today’s episode, our conversation centers on her book, and her perspective on the emerging patterns of cloud native software.

Python is one of the most popular programming languages in the software world. After working with Python and developing a love for the language, Michael Kennedy started to wonder why there was not a high quality podcast dedicated to covering the community and new technologies of the Python ecosystem. Michael started Talk Python To Me as a podcast with the goal of telling stories from the world of Python. Today, Talk Python To Me is the most popular podcast dedicated to Python and related technologies. Subjects on the podcast include web frameworks, compilers, career development, and cloud services. One of the more recent developments in the world of Python is the prevalence of data science, which Michael also covers in great detail. Michael has also spun up a second podcast called Python Bytes, which offers timely updates to the Python community. Michael joins the show to share his thoughts on several topics related to Python, including compilers, data science workflows, and web frameworks. Michael also gives his perspective on the world of software podcasting, which he has been doing for more than four years.

A new software application has simple requirements for a database. The database needs to be written to and read from. The database fulfills simple needs such as storing user information and providing the application frontend with the necessary data to render a simple webpage of financial transactions or blog posts. As an application becomes successful, the database grows in size. The complexity of queries increases, requiring more sophisticated logic in order to maintain performance. New databases need to be added to the overall system, as users begin to have demands for advanced features such as search, or analytics. Over time, the requirements for a database expand into a need for a data platform. A data platform might include multiple databases such as a NoSQL database, a relational database, a data warehouse, and a search index. The relationships between these different databases can vary in terms of consistency requirements, latency, and scalability. Andrew Davidson is the Director of Cloud Products at MongoDB. In a previous episode, Andrew discussed the tradeoffs of scaling databases while maintaining high performance indexing. Andrew returns to the show to discuss the emerging subject of “data platform.” As a growing number of companies have data requirements beyond that of a simple transactional database, Andrew’s work has increasingly involved figuring out the best ways for developers to adapt those transactional systems to providing a wider set of functionality. Full disclosure: MongoDB is a sponsor of Software Engineering Daily.

Spanner is a globally distributed, transactionally consistent database. Spanner initially emerged as a paper that came out of Google in 2012. Around this time, database scalability was difficult to solve, even for Google. The Spanner paper offered some breakthroughs in distributed systems which allowed Google to take some of the learnings from BigTable’s eventual consistency and construct a novel system that was transactionally consistent. Deepti Srivastava has worked on Spanner for more than six years. When she initially started working on Spanner, she was involved in the internally facing Spanner system that Google engineers used to spin up instances of the database. Today, she works as program manager for Cloud Spanner, a product within Google Cloud. Deepti joins the show to describe the breakthroughs of Spanner, and how her work has evolved since the product has gone from an internal system to an externally facing piece of cloud infrastructure.

Open source plays a key role in today’s world of technology businesses. Today, the impact of open source seems obvious. From Kubernetes to distributed databases to cloud providers, so much of our software is powered by open source. But it was not always this way. Bruce Perens was one of the earliest figures in the world of open source. He collaborated with Eric S. Raymond, the author of Cathedral and the Bazaar. He developed a differing belief set from Richard Stallman, who was another foundational advocate of open source. Bruce worked as an engineer at Pixar from 1987 to the year 2000. Bruce joins the show to give a history of open source and his experience in the software industry.

The browser has become the central application of the consumer operating system. Every piece of client software, from email to document management, has become usable through the browser. Even modern desktop software such as Slack is built using Electron, a tool for building client applications that essentially run inside of a browser without an address bar. One activity that still takes place largely outside of the browser is the process of writing and deploying code. A developer often uses an IDE such as Eclipse to write their code, then switches over to a terminal where they can build and deploy their code to a remote server running in the cloud. For a developer who has been writing code for a long time, this process feels completely intuitive. But for a new developer, it can be totally disorienting. New developers sometimes have trouble understanding the difference between a local and remote environment, or how to use repository management software like Git. This is in addition to all the other problems a new developer might be dealing with, such as language installation, syntax, and package management. Repl.it is a browser-based coding environment, compute engine, and collaborative workspace. Repl.it has found significant traction among new programmers who begin their programming journey within Repl.it and then stay in the environment, even as their applications become more richly featured and complicated. Repl.it is an amazing piece of software, and the story behind it is remarkable. Amjad Masad had the idea for Repl.it many years before he started the company for it, but he needed to first build up the money and confidence in order to go after the business with full force. Amjad joins the show to talk about his long journey towards building Repl.it, and to discuss the thriving Repl.it platform in its current form.

Modern software is built with cloud services, APIs, and other high level tools. Technical software development is moving beyond the realm of writing code. Individuals who do not have a background in computer science or programming can create increasingly complex tools. Higher level APIs include Twilio for managing phone communications, and Stripe for managing financial workflows. Platforms such as Shopify can be used as the core system for building an e-commerce business. Low-code tools such as Wix can be used for building web sites. Monday.com was started in 2012 with the goal of making software for managing business workflows. Business workflows link together different teams and processes within a company. Monday allows non-engineers to automate workflows that are traditionally automated by engineers. For example, Monday can be used to build workflows that link together SalesForce and Twilio, or to trigger actions in MailChimp based on how a user interacted with a Shopify web site. One of the biggest developing trends in technology is how non-engineers are becoming empowered to have more impact within an organization. The advances in these kinds of tools present us with an opportunity to rethink team structures, and invent entirely new roles for a software company. Eran Zinman is the CTO and co-founder of Monday.com and he joins the show to describe the vision for Monday and how the product fits into the changing enterprise software trends around APIs and “low code” tools. Monday has grown exponentially over the last 7 years, with the most recent growth curve looking almost vertical. Eran’s experience scaling the product and improving performance makes for excellent storytelling. Full disclosure: Monday is a sponsor of Software Engineering Daily.

Cassandra was initially released in 2008 as a project out of Facebook. Cassandra offered an open source solution to database scalability issues that were being tackled internally by large companies like Amazon, Google and Facebook. 2008 was a golden age of new infrastructure, with systems such as Hadoop and Kafka gaining traction around the same time. Jonathan Ellis started working with Cassandra, and became intrigued by the system. With the help of investors, Jonathan began working on a company based around Cassandra called Datastax. Today, Datastax is a company with more than 450 employees and a large valuation. Jonathan joins the show to discuss his experience working with Cassandra, and his reflections on the becoming an entrepreneurial founder of a highly successful database company. There is a wealth of useful knowledge for both software engineers and entrepreneurs in this episode.

Airlines have always had an emphasis on new technology. Over the years, airlines have needed to develop more and more software. Digital transformation is causing every large company to adopt the tools and practices of software companies, and that includes Delta Airlines. Delta Airlines has existed for more than 90 years. Over that period of time, the company has developed new systems in every generation of software, from the days of mainframe computers to a modern Java-based backend. When the DevOps movement started to take shape, Delta Airlines started to take a more focused approach on continuous integration, version control, and an organizational structure that removed silos between teams. Jasmine James is a manager at Delta Airlines where she focuses on improving the software practices of the company. She joins the show to talk through the process of changing the developer culture within Delta, as well as what it is like to build software for an airline. Jasmine is speaking at GitLab Commit in Brooklyn on September 17, 2019, and GitLab is a sponsor of Software Engineering Daily, so if you are thinking about attending GitLab Commit, you can go to softwareengineeringdaily.com/commit and enter code COMMITSED to save 30% on your ticket, while also supporting Software Engineering Daily.

Ilya Sukhar founded Parse in 2011 to make cloud services for mobile developers. Parse was a newer kind of cloud provider called “backend-as-a-service”, built to simplify the complexities of Amazon Web Services and the complexities of the mobile ecosystem. During this time, Facebook was in the middle of its shift toward becoming a mobile application company. Ads on the smartphone were not yet a proven business model. Facebook was exploring other business lines, and decided to purchase Parse for $85 million, with the intention of building a cloud developer platform. Shortly after the acquisition, Facebook’s mobile ads business started to see considerable success. With the mobile ads business finding traction, Facebook shifted all available resources towards supporting that business model. In 2017, Parse was shut down. Ilya joins the show to give his experience starting Parse, selling the company to Facebook, and then seeing the company he had built get shut down as it became an unfortunate casualty of Facebook’s advertising success. We talked a lot about the experience of building a backend-as-a-service company, as well as what makes Facebook special as an organization, and how the success of Facebook’s mobile business happened to coincide with the Parse acquisition.

Parse was a backend-as-a-service company that started in 2011 with the initial focus of making the cloud easier to use for mobile developers. Parse had several novel engineering challenges. In 2011, it was not easy to build on top of AWS, nor was it easy to build within the young mobile ecosystem. Charity Majors was an early engineer at Parse, and stayed with the company through its acquisition by Facebook. Charity joins the show to discuss her experience at Parse and her reflections on being part of Facebook after the acquisition. Parse was shut down in 2017, which was a disappointing outcome for Charity. Charity currently works as the CTO of Honeycomb. In a previous episode, Charity gave some background on her career in software, as well as her perspective on modern problems in observability and DevOps. In this episode, Charity tells the story of joining Parse, working through novel engineering problems, and the feeling of being acquired by a company where she did not feel like a great cultural fit. It is a useful conversation for anyone interested in engineering culture, company philosophy, and the realities of acquisitions–which can be both painful and profitable for the acquired employees.

Kent Beck is a legendary figure in the world of software engineering. Kent was an early advocate of Test-Driven Development (TDD), and popularized the idea of writing unit tests before writing code that would satisfy those unit tests. A unit test isolates and tests a small piece of functionality within a large piece of software. Practitioners of Test-Driven Development write tens or hundreds of tests in order to cover a large variety of cases that could potentially occur within their software. When Kent Beck joined Facebook in 2011, he was 50 years old and thought he had seen everything in the software industry. During Facebook Boot Camp, Kent started to realize that Facebook was very different than any other company he had seen. Facebook Boot Camp is the six-week onboarding process that every new hire learns about the software practices of the company. After graduating Facebook Boot Camp, Kent began to explore Facebook’s codebase and culture. He found himself rethinking many of the tenets of software engineering that he had previously thought were immutable. Kent joins the show to discuss his time at Facebook, and how the company’s approach to building and scaling products thoroughly reshaped his beliefs about software engineering.

When Chuck Rossi joined Facebook in 2008, he was one of the most experienced release engineers at the company. As he began to explore the engineering practices of the organization, he was surprised, confused, and impressed by the release engineering system that he encountered. Release engineering is the process by which software is released to users. As software is being developed, it moves through a series of testing environments. In these test environments, the software can be studied using simulated inputs that can help developers discover software bugs. Chuck had come to Facebook from Google. At Google, the crown jewel was Google web search which had a regimented release process. At Facebook, the crown jewel was facebook.com. Chuck found that the release process for facebook.com was much different than Google web search. Chuck joins the show to talk about release engineering at Facebook, and how the company constantly evolved its code deployment process. Chuck also describes Facebook’s pivot to mobile, and how the bottlenecks in the mobile app release process threatened Facebook’s ability to iterate and release quickly. This show provided some amazing perspective on continuous delivery, and will be useful to anyone who is working on figuring out their “DevOps” process. Chuck has a wealth of knowledge and context about the modern software industry.

Facebook is a large multiuser application. Scaling Facebook was different than scaling a single-user application such as an ecommerce store or a search engine. A social network is faced with unique infrastructure scalability challenges, as well as subjective questions around user communications, privacy, and content. Pedram Keyani worked at Google before joining Facebook in 2007. In his years at Facebook, he worked on infrastructure, internal tools, and management. He became deeply familiar with the company culture and its operations. Pedram joins the show to talk about how Facebook has scaled and the lessons he took away from his time there. After his time at Facebook, Pedram joined Uber, where he worked as a director of engineering for four years. Uber is another multi-user application, with a very different set of constraints. At Uber, Pedram worked on several projects, including Uber’s push into China, which he describes as an intense, competitive experience. Pedram is able to contrast the culture and scaling processes of Uber, Facebook, and Google which made this a rare opportunity to see how three different high performing companies build software differently.

Cryptocurrencies are decentralized monetary systems built on open source software. The open source software movement has evolved from the world of Linux, MySQL, and Apache to a thriving ecosystem of commercial enterprises built around open source software. This ecosystem includes projects such as Kubernetes, MongoDB, and ReactJS. It includes large organizations such as Amazon Web Services, Elastic, and Facebook. In a parallel software universe, the crypto ecosystem has built revolutionary new financial tools, creating billions of dollars of value, but not very many massive commercial companies. In the world of cryptocurrencies, many of the same rules apply to the classic open source world. But other rules do not. How do these two worlds differ from each other? How are they the same? And how might they end up colliding? Haseeb Qureshi, Joseph Jacks, and Alok Vasudev join the show for a spirited discussion of cryptocurrencies and open source. Haseeb is a cryptocurrency investor, JJ is the founder of OSS Capital, and Alok is an engineer and the founder of crypto venture capital firm Standard Crypto.

When a new employee joins a software company, it is often unclear where that employee should begin. Do they have a mentor? What are they working on? What are the expectations for how fast that employee should be contributing? The early period of employment is often referred to as “onboarding.” During the onboarding period, an employee will learn the basics of the company they have just joined. These basics include the work procedures, meeting schedules, expectations, and technical tools that an employee needs to become productive. A poor onboarding process can slow an employee down, and even cause them to quit the company entirely. Conversely, a good onboarding process can accelerate an employee towards rapid contribution within the organization. A scalable onboarding process can be a difference in millions of dollars in productivity per year across the organization. Kristen Gallagher is the founder of Edify, a company that helps organizations define and implement their onboarding process. Kristen joins the show to talk about the ingredients of a successful onboarding process and what she is doing with her company Edify.

A time series database is optimized for the storage of high volumes of sequential data across time. Time series databases are often organized as columnar data stores that can write large volumes of data quickly. These systems can sometimes tolerate data loss, because the data they are gathering is used for monitoring and other applications that require aggregated data sets rather than highly important individual transactions. The demand for time series databases has grown over the last decade with the rise of mobile devices and the decreasing cost of cloud storage. There has been an increase in the number of systems that require monitoring, and some of those systems produce an incredibly large amount of data, requiring compression, downsampling, and garbage collection. Rob Skillington is an engineer at Uber, where he helped create M3DB, a time series database. In a previous show, Rob described the basics of M3DB and how it helps Uber with storing data from Prometheus, a monitoring system. In today’s show we discuss the field of time series databases, and Rob’s approach to building M3.

Insurance is an old business. Individuals and businesses have been buying insurance policies for decades. These insurance policies can cost hundreds, thousands, or tens of thousands of dollars per year. Software is remaking the insurance industry. One way new software can improve the insurance industry is through better brokerage technology. Insurance involves carriers and brokers, who work together on delivering insurance solutions to customers. The initiation and closing of an insurance transaction often involves lots of emails, PDF files, and antiquated software systems. Technology improvements will smooth out this process and reduce manual overhead. Another way insurance can improve thanks to technology is through smarter pricing. The price of an insurance policy is offered to a customer based on how risky the insurance policy is, how large the customer pool is, and how much the insurance company could lose in the event that it would have to pay out an insurance policy. These risk profiles are calculated based on historical data. Gordon Wintrob is the co-founder and CTO of Newfront Insurance. Gordon joins the show to discuss the insurance industry and how his company got started.

Open source software has evolved into a thriving, multifaceted ecosystem. Open source encompasses operating systems and databases. Open source embodies both altruism and self-interest. And open source enables thriving businesses from WordPress blogs to hundred billion dollar cloud providers. There is a large set of business models that can be built around a successful open source project. Some of these business models are more defensible than others. A company based around open source must make a deliberate set of tradeoffs around how their private company will relate to the public open source ecosystem. Today’s episode is a discussion of open source business models with Karthik Ranganathan, Heather Meeker, and Matt Asay. Karthik is the CTO of YugaByte, an open source distributed SQL database. Heather Meeker is an open source licensing specialist and a founding partner of OSS Capital. Matt Asay is a longtime executive who has worked in several open source based companies including MongoDB–he has also written at length about open source. Full disclosure: YugaByte, where Karthik works, is a sponsor of SE Daily.

Andreas Antonopoulos is the author of several books about cryptocurrency engineering, including Mastering Bitcoin and Mastering Ethereum. In these books, Andreas lays out the systems of economics and computer science that underpin the two most mature decentralized monetary systems. When Andreas originally discovered the Bitcoin whitepaper, he had witnessed the repeated mismanagement of government-backed fiat currencies. Andreas has a Greek background, and the financial collapse of 2008 had led to an economic crisis in Greece. His firsthand observation of the weaknesses of centrally planned government currencies, together with a degree in computer science and distributed systems have made him a dedicated evangelist for Bitcoin. Andreas joins the show to discuss the Bitcoin ecosystem, and the relationship between decentralized cryptoeconomic systems and centralized corporations. Facebook has recently announced a cryptocurrency project called Libra, and Andreas suggests that Libra changes everything–not necessarily because Libra will make it to production, or because Libra itself will upend the world of finance–but because it allows us to further call into question the very nature of what makes a modern currency valuable and valid. After all, if a large government has the right to back a currency, why shouldn’t a large corporation have that same privilege? Andreas is also a co-host of one of my favorite Bitcoin podcasts, Let’s Talk Bitcoin.

Software engineers often work as a contractor for some duration of their career. A contractor earns a fixed hourly salary for a defined period of weeks, months, or years. Contract work can be more flexible than full-time work, and often pays more than full-time software engineering, because contract jobs can end at any time, and they do not have the added employee benefits such as health insurance and stock options. Online contracting platforms such as Upwork and Fiverr have expanded the number of software contracting engagements that take place. Developers on Upwork and Fiverr have a wide range of skills and experience levels. The clients who come on Upwork and Fiverr looking for developers are sometimes unsophisticated at managing software projects, and in some cases they defraud the software developers who are making their living online, and trick these software developers into delivering free work. Moonlight is a contracting platform for software engineers. Today’s guests Emma Lawson and Philip Thomas are the founders of Moonlight, and they join the show to explain why they started the company, and the gaps that exist in the world of software contracting. Moonlight’s model is different than most other contracting platforms, in that Moonlight requires clients to pay a $300 subscription fee to recruit engineers on the platform. This $300 subscription price lowers the rate at which clients take advantage of software engineers. This dynamic causes the software engineers to take their work more seriously and act more professionally. Full disclosure: I am an investor in Moonlight. I have also been a paying client of the service, and our newest version of the Software Daily app was built by Mostafa Gazar, who is a talented Android engineer I met on Moonlight.

The service mesh abstraction allows for a consistent model for managing and monitoring the different components of a microservices architecture. In the service mesh pattern, each service is deployed with a sidecar container that contains a service proxy. These sidecars are collectively referred to as the “data plane.” Each sidecar provides the service that it is deployed next to with a set of features such as security policy, rate limiting, and monitoring instrumentation. The sidecars in the data plane communicate with a central module called a control plane. In the control plane, an engineer can operate across these individual services at scale, by pushing out updates to them. Kubernetes has made it easier to manage large fleets of microservices, and has led to wider adoption of service mesh. Istio is one of the most popular service mesh products. In today’s show, Varun Talwar returns to the show to describe the state of the Istio project and the process of deploying Istio to a cluster. Varun is the CEO of Tetrate, a company building an enterprise-ready service mesh. Prior to Tetrate, Varun was at Google, where he helped found the gRPC and Istio projects.

The PlayStation is a line of game consoles created by Sony. PlayStation devices include the PS2, PS3, PS4, and the PSP mobile system. Tony Godar worked as an engineer in the PlayStation ecosystem for 15 years, and he joins the show to give a retrospective on his time in the console industry. Developing hardware and software for game consoles differs significantly from the world of web development. Tony describes the culture of the game development world, and the challenges involved in the domains of software tooling, custom operating systems, and streaming media. In 2010, the PS3 was hacked by notorious tinkerer George Hotz, a previous guest on the show, an event which Tony also discusses. We also discuss the world of modern gaming and VR technology. Tony currently works as an engineer at MelodyVR, a company that makes virtual reality live music experiences.

Large software companies have become a target for criticism. Google, Facebook, Amazon and other prominent technology giants find themselves under a kind of scrutiny that is reminiscent of banks in 2008 and oil companies in the early 1900s. Across the planet, there is a growing sentiment that “big tech” has too much power, and that they are abusing that power in order to dominate markets, censor speech, and manipulate politics. Tyler Cowen is the author of Big Business: A Love Letter to an American Anti-Hero. He is also the host of the popular podcast Conversations with Tyler and an economist at George Mason University. In Big Business, Tyler explores the modern relationships between consumers and businesses, including the large technology companies. Tyler joins the show to discuss his new book. In previous episodes with Tyler, we also talked about his previous books as well as the effects of technology on American society and the philosophical approaches software engineers can bring to their careers.

In this episode Will Larson, author of An Elegant Puzzle: Systems of Engineering Management, speaks with Uma Chingunde of Stripe and Jeff Meyerson of Software Engineering Daily about engineering management. Will was also featured on SE Daily recently. An Elegant Puzzle is an excellent resource on management techniques and strategies for scaling software organizations. Will’s writing draws from his experiences working at Uber and Stripe.

The a16z Podcast is a show that is produced by Andreessen Horowitz, an investment fund based in Silicon Valley. The a16z Podcast covers topics including software engineering, biology, media, cryptocurrencies and entrepreneurship. A16z is one of the most popular podcasts about technology. Sonal Chokshi is the editor in chief at Andreessen Horowitz and the showrunner for the a16z podcast. For five years, she has been interviewing entrepreneurs, engineers, artists, and investors, exploring how software has increasingly impacted our lives and transformed society. The success of the a16z Podcast is largely a result of Sonal’s high editorial standards and her ability to ask the right questions and drive conversations in fruitful directions. Much of the content of Software Engineering Daily has been shaped by a16z, and I have listened to every single episode. Sonal Chokshi joins today’s show for a conversation about podcasting and technology. Sonal shares her beliefs for why the podcast medium has taken off, and describes how her background in education, ethnography, and technology has shaped the completely distinct voice and flavor of the a16z Podcast.

Data-as-a-service businesses offer paid access to data sets. These data sets can be useful for building products or training machine learning models. There has been steady growth in the tools and practices around processing and storing data. But access to data sets remains a bottleneck for widespread development of machine learning applications in a large set of domains. SafeGraph is a company focused on the problem of data-as-a-service. Today, SafeGraph’s primary product is reliable, up-to-date location information on places. The data for these points-of-interest needs be acquired, verified, cleaned, made accessible through an API, and intelligently priced. In previous episodes with SafeGraph, we have explored the basic premise of data businesses and why they are important platforms for building futuristic data products that are impossible for entrepreneurs to build today. SafeGraph CEO Auren Hoffman returns to the show to discuss the data as a service business model. Software-as-a-service has existed as a category for more than a decade. Infrastructure-as-a-service has existed for just as long. Data-as-a-service is much more undeveloped. Auren recently published the “Data-As-A-Service Bible: Everything You Wanted To Know About Running A DaaS Business”. This was a very useful article, as it breaks down a category of software that is almost entirely unexplored.

A large sales organization has hundreds of sales people. Each of those sales people manages a set of accounts who they are trying to close sales deals on. Sales people are overseen by managers who ensure that the sales people are performing well. Directors and VPs ensure the scalability and health of the overall sales organization. The sales lifecycle mostly takes place within a piece of software called a CRM: customer relationship management. This tool documents the interactions between sales people and accounts. CRMs have been around for many years, and although CRM software is a useful repository of data, it does not fulfill all the needs of a salesperson. People.ai is a system of machine learning tools built around the sales tooling ecosystem. People.ai helps a sales organization avoid manual data entry, understand areas of potential improvement, and decide on who the highest value sales lead to pursue might be. Andrey Akselrod is the CTO At People.ai and he joins the show to discuss the potential applications of machine learning in the domain of sales, and the engineering work that his company has done.

During 2015, Uber was going through rapid scalability. The internal engineering systems were constantly tested by the growing user base. Over the next two years, the number of internal services at Uber would grow from 500 to 2000, and standardizing the monitoring of all these different services became a priority. After working with a variety of available tools, Uber’s engineering team decided that something new needed to be built internally. Jaeger is an open source distributed tracing tool that provides observability features throughout Uber’s microservices architecture. Yuri Shkuro is an engineer at Uber, where he works on Jaeger and other infrastructure projects. He joins the show to discuss the history of engineering at Uber, the architecture of Jaeger, and the requirements for building and scaling a distributed tracing tool.

A knowledge base assembles information from a wide variety of sources into a central platform. The most popular knowledge base is Wikipedia, which covers a wide variety of concepts through a system that attempts to remain authoritative and impartial. Other open knowledge platforms include Stack Overflow, which focuses on programming concepts, and Quora, which adds a social element to the process of information accumulation. Golden.com is a knowledge base that indexes, categorizes, and surfaces information. Golden has information about software, genetics, world history, social media, sports, and all kinds of other subjects. The company monetizes with a paid knowledge base product for enterprises, which allows for easy easy querying of the open knowledge base and a private knowledge base for internal information. Jude Gomila is the founder of Golden, and joins the show to discuss the process of building a universal knowledge base, and the engineering problems that he is working on to improve Golden.

Coding bootcamps allow anyone to become a programmer at a faster pace than the traditional computer science education system. In the last five years, coding bootcamps have grown rapidly in popularity, with thousands of people gaining the necessary skills to work as a software engineer. Career Karma is a platform that allows individuals to find the best coding bootcamp. There are many coding bootcamps, and they are not all the same. Much like different schools have different cultures and focus on different disciplines, coding bootcamps vary widely in their teaching styles and acceptance path. Reuben Harris and Artur Meyster are co-founders of Career Karma, and they join the show to discuss the changing nature of software engineering education and the frictions that new programmers encounter as they navigate the world of coding bootcamps. They also describe their journey to entrepreneurship and their own personal experience with coding bootcamps.

A hackathon is an organized event where participants work together to build a product or tool. Hackathons are about creativity, learning, and exploration. A developer that is participating in a hackathon is often working on something that is outside of their normal day-to-day focus. Hackathons can provide significant value to the participants. Hackathons have led to friendships, new companies, and newly developed confidence that can be critical to a developer who feels uncertain in their ability to launch their own projects. Jonathan Gottfried is a co-founder of Major League Hacking, an official student hackathon league that powers invention competitions. Major League Hacking is a B corporation that is focused on improving the education and community of technology leaders, entrepreneurs, and young hackers. Jon joins the show to discuss hackathons–and how he has built and scaled an organization that is devoted to creating systematically successful hackathon experiences.