PLAY PODCASTS
Software Engineering Daily

Software Engineering Daily

2,200 episodes — Page 24 of 44

Ep 1197TechPrimers: Software Engineering YouTube with Ajay Kumar

Videos play a key role in the continuing education of a software engineer. Video can capture many different types of content that is useful for engineers: conference talks, tutorial videos, and podcast-style interviews are all popular formats of online video. YouTube has become the predominant source for video content about software engineering. The open nature of YouTube’s format allows for a tremendous range of content. No matter what your preference is for how you like to learn and be entertained, YouTube has something for you. TechPrimers is a media channel that is dedicated to sharing technical knowledge in the form of videos, GitHub repositories, and a thriving community. TechPrimers has over 300 videos on YouTube, and more than 48,000 subscribers who regularly watch content about subjects like AWS, Spring, and Kubernetes. TechPrimers was founded by Ajay Kumar, a software engineer and vice president at JP Morgan. Ajay has been in the software industry for fifteen years, and has been working in enterprise banking systems for seven years. He has deep experience in modern technologies and engineering practices. Ajay joins the show to discuss the modern world of software engineering, and his experience building a media platform. He also talks about the technology culture of India. Ajay is based in Bengalaru, and it was exciting to learn how much our different societies have in common thanks to technology. Ajay also had me on his YouTube channel for an interview, which was a lot of fun.

Jul 31, 20191h 0m

Ep 1196WebAssembly on IoT with Jonathan Beri

“Internet of Things” is a term used to describe the increasing connectivity and intelligence of physical objects within our lives. IoT has manifested within enterprises under the term “Industrial IoT,” as wireless connectivity and machine learning have started to improve devices such as centrifuges, conveyor belts, and factory robotics. In the consumer space, IoT has moved slower than many people expected, and it remains to be seen when we will have widespread computation within consumer devices such as microwaves, washing machines, and lightswitches. IoT computers have different constraints than general purpose computers. Security, reliability, battery life, power consumption, and cost structures are very different in IoT devices than in your laptop or smartphone. One technology that could solve some of the problems within IoT is WebAssembly, a newer binary instruction format for executable programs. Jonathan Beri is a software engineer and the organizer of the San Francisco WebAssembly Meetup. He has significant experience in the IoT industry, and joins the show to discuss the state of WebAssembly, the surrounding technologies, and their impact on IoT.

Jul 30, 201952 min

Ep 1195Data Mesh with Zhamak Deghani

Data engineering involves numerous tools–a data lake, databases, data warehouses, numerous APIs, streaming systems, and microservices. There is no shortage of ways to interact with data and manage data, but many companies are struggling to figure out design patterns and best practices for how to manage data and build data infrastructure. Zhamak Dehgani is a principal consultant and portfolio director with ThoughtWorks, where she works with enterprises to improve their software systems and workflows. She is the author of an article called How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh and she joins the show to discuss her perspective on data infrastructure, as well as the “data mesh” concept that she has coined. Data mesh represents the movement away from having a centralized data lake that all teams interact with, and towards having different data products and individual data management systems for individual domain teams.

Jul 29, 201954 min

Ep 1194Software IPOs with Tomas Tunguz

Software companies such as Slack, Zoom, and Uber have recently gone public. When a company goes public, they issue a document called an S-1. Within the S-1, there is a wealth of information about the company, providing a detailed story about the company’s business model, economics, and future prospects. The S-1 describes the operating model and the philosophy of a newly public company. Going public serves several purposes. Being public allows a company to gain access to the public capital markets. It allows previous investors to have a liquidity event, by selling the shares that they purchased from the company in private markets. Being public also puts some constraints and visibility on a company, which can be useful for a company that is trying to develop internal discipline. In the software industry, it is useful for most people to understand the dynamics of going public. A technology worker who is earning equity at a private company needs to understand the roadmap to their company going public, or potentially getting acquired. Anyone who invests in public technology stocks is evaluating the different available options for investment, and considering the best software companies to place a bet on. Tom Tunguz is a venture investor at Redpoint, and the author of a popular blog at tomasztonguz.com. In a recent series of posts, Tom has evaluated the S-1s and compared the growth dynamics between a variety of newer public software companies. Tom joins today’s show to discuss his writing, and offer reflections on what can be learned about company building from the recent series of IPOs and direct listings.

Jul 26, 201956 min

Ep 1193Envoy Mobile with Matt Klein

Envoy is an open source edge and service proxy that was originally developed at Lyft. Envoy is often deployed as a sidecar application that runs alongside a service and helps that service by providing features such as routing, rate limiting, telemetry, and security policy. Envoy has gained significant traction in the open source community, and has formed the backbone of popular service mesh projects such as Istio. Envoy has been mostly used as a backend technology, but the potential applications of Envoy include frontend client applications as well. The goal of Envoy is to make the network easier to work with–and the network includes client applications such as mobile apps running on a phone. Envoy Mobile is a network proxy for mobile applications. Envoy Mobile brings many of the benefits of Envoy to the mobile client ecosystem. It provides mobile developers with a library that can simplify or abstract away many of the modern advances that have been made in networking in recent years, such as HTTP2, gRPC, and QUIC. Matt Klein is the creator of Envoy, and he joins the show to discuss Envoy Mobile. Matt describes how the networking challenges of mobile applications are similar to those of backend systems and cloud infrastructure. We discuss the advances in networking technology that Envoy Mobile helps bring to the mobile ecosystem, and also touch on the scalability challenges that Matt is seeing at Lyft.

Jul 25, 201957 min

Ep 1192TechLitAfrica: Computer Literacy in Africa with Tyler Cinnamon

In the developed world, it is easy to take for granted that we grew up with computers. Technology is so pervasive in the United States that we have debates about how early in child development a human should be given a smartphone. Across much of Africa, there is a shortage of access to computers. Children grow up without much exposure to computers at all. Smartphones are starting to proliferate the continent, but the bandwidth limitations prevent the sort of unrestricted mobile Internet usage that many of us have in the west. Tyler Cinnamon is a software engineer and the co-founder of TechLitAfrica, an organization dedicated to improving technology literacy and reducing poverty in Africa. TechLitAfrica takes old computers from the United States which are no longer in use and repurposes those computers with educational software and a downloaded subset of the Internet. Then, TechLitAfrica takes those computers to Africa and sets them up as computer labs. Tyler joins the show to talk through the technical and cultural challenges of building TechLitAfrica.

Jul 24, 201945 min

Ep 1191Data Engineering with Tobias Macey

The Hadoop ecosystem provided every company with the tools to store and query large amounts of data at a low cost. Since 2005, that ecosystem has expanded with more and more open source applications for data infrastructure. Data infrastructure includes databases, data lakes, distributed queues, data warehouses, query engines, web applications, on-prem software, closed source, open source, cloud platforms, and software-as-a-service. There is so much software, and it is difficult to keep track of everything in the data engineering ecosystem. Tobias Macey is the host of the Data Engineering Podcast, a show about technologies and practices in the world of data engineering. Tobias joins today’s episode as a guest to give his perspective on the evolving landscape of data engineering, and the trends he is seeing on the data engineering podcast.

Jul 23, 20191h 4m

Ep 1190Emerging Markets: Vietnam with Charles Lee

From Africa to India to Asia to South America–computer science and programming are rising in popularity in every emerging market. Each of these markets has regional needs for technology. Just like every culture develops its own food and television, every culture needs different types of applications to run their lives. In Vietnam, the day-to-day life of a citizen is different than it is in the United States. Yes, everyone needs Google and YouTube and Instagram. But the trends in messaging, food delivery, gig economy, and other B2C technology sectors are considerably different than the west. Charles Lee is the founder of CoderSchool a coding school in Vietnam. Before moving to Vietnam to start CoderSchool, he worked as a software engineer in San Francisco for several years. In today’s show, Charles describes the difference between the US and Vietnamese technology sectors, from consumer applications to business software to coding education.

Jul 22, 201957 min

Ep 1189Facebook GraphQL with Lee Byron

In 2011, Facebook had begun to focus its efforts on mobile development. Mobile phones did not have access to reliable, high bandwidth connections, and the Facebook engineering team needed to find a solution to improve the request latency between mobile clients and the backend Facebook infrastructure. One source of latency was recursive data fetching. If a mobile application client made a request to the backend for newsfeed, the backend API would return the newsfeed, but some components of that feed would require additional requests to the backend. In practice, this might result in a newsfeed loading partially on a phone, but having a delayed loading time for the comments of a newsfeed item. GraphQL is a solution that came out of this problem of recursive data fetching. A GraphQL server provides middleware to aggregate all of the necessary information to serve a complete request. GraphQL connects to backend data sources and federates the frontend request across these different data sources. GraphQL was open sourced in 2015, and has found many use cases in addition to simplifying backend data fetching for mobile clients. Today, GraphQL is used by PayPal, Shopify, Twitter, and hundreds of other companies. Lee Byron is the co-creator of GraphQL and he joins the show to tell the story of GraphQL, and how it fit into Facebook’s shift to mobile.

Jul 19, 201954 min

Ep 1188Facebook Open Source Management with Tom Occhino

Facebook has released open source software projects that have changed the industry. The most impactful projects to date are the React frontend user interface tools: ReactJS and React Native. Before React became popular, there were multiple competing solutions for the dominant frontend JavaScript framework. React became the most prominent because of its invention of JSX, its one-way data flow, and the strength of its community. The React community is led by Facebook engineers. Facebook has a full-time team dedicated to improving React. Facebook also has the benefit of seeing the hardest problems in web development ahead of the rest of the industry, since Facebook is always pushing the leading edge of what software can do. As React evolves, Facebook itself is the first user of updates to the libraries. Engineers on the React team will proactively update the applications on other teams with new React code. In this way, Facebook’s React team is similar to a platform engineering team, except the open source community benefits from the improvements as well. Tom Occhino is an engineering director at Facebook. He joins the show to discuss his ten years of experience at the company and his role today as a senior leader in charge of open source developers.

Jul 18, 201958 min

Ep 1187Facebook Data Infrastructure with Dhruba Borthakur

Facebook generates high volumes of data at a rapid pace. Dhruba Borthakur joined Facebook in 2008 to work on data infrastructure. His early projects at Facebook were around Hadoop, the distributed file system and MapReduce computation platform that laid the foundation for the “big data” movement. At the time, Facebook was generating as much data as any other startup, and the company needed to stay at the leading edge of scalability techniques for its Hadoop Distributed File System (HDFS) cluster. Traditionally, Hadoop managed its file system by synchronizing the coordination of the different data nodes with the help of a single master node. At Facebook, the scale of the data was such that the HDFS cluster had thousands of data nodes, which was too much volume for a single master node to handle. Dhruba helped implement redundancy at the master node to create a more resilient system. The early days of the big data movement was focused on batch processing. A company like Facebook would gather large amounts of data into databases and HDFS, and run offline analytics workloads to gather reports on an hourly, daily, or weekly basis. Over time, data infrastructure has moved closer to a “real time” processing model. Data infrastructure does not only support batch offline reporting–it also supports machine learning jobs that need to be run on a more frequent basis. These jobs have lower latency requirements, and have driven the adoption of in-memory stream processing systems like Spark and Flink. Dhruba joins the show to discuss his time at Facebook building data infrastructure. He takes us through the major projects he worked on, including the early Hadoop infrastructure, the refactoring of online user workloads to be more “pull” based than “push” based, and the creation of RocksDB, a storage engine he helped create at Facebook. Today, Dhruba is the CTO and co-founder of Rockset, a company that builds data infrastructure and database APIs on top of RocksDB. Rockset is building infrastructure for modern technology companies–many of which are facing problems that bear significant resemblance to the ones Facebook encountered as it scaled.

Jul 17, 201932 min

Ep 1186Facebook Engineering Culture with Raylene Yung

Facebook moves fast because of vision, collaboration, and trust. The fast pace of development is enabled by constantly improving infrastructure and a sense of unity throughout the company. In Facebook’s early days, there was an emphasis on rapidly deploying new code to drive constant improvement and experimentation. Product quality was maintained by engineers closely checking each other’s code reviews, rather than writing detailed unit testing suites. Facebook engineers had a sense for how the product should operate, and they were able to evaluate whether a new feature was working properly by testing a live version of that feature. At Facebook, the vision of the company is clearly communicated to the employees. Every employee within Facebook can articulate the vision for the company, and will use similar language in describing that vision. Since the employees are aligned on strategy, they can also align in their implementation of product features. This reduces conflicts across roles and between teams. Facebook has also shown a willingness to trust its engineers. Trust was exemplified by Facebook’s tolerance for failures in the early days. When an engineer broke a build, or shipped a feature that failed to gain traction, that engineer was usually not punished. They may have even been rewarded, if the company could learn significantly from such an error. Raylene Yung was an engineer at Facebook from 2009 to 2015. As she moved from individual contributor to manager to engineering director, Raylene worked on products including News Feed, Timeline, Privacy, and Sharing. Raylene joins the show to give her reflections on the Facebook product and engineering environment. She explained how Facebook’s culture of collaboration, vision, and trust drive fast product development and minimizes conflict. Raylene left Facebook and joined Stripe, where she worked on payments systems and international expansion for almost four years.

Jul 16, 201942 min

Ep 1185Facebook PHP with Keith Adams

Facebook was built using PHP, a programming language that was used widely in the late 90s and early 2000s. PHP allows developers to get web applications built quickly and easily, although PHP has a reputation for being difficult to scale. In the early days of Facebook, the company was scaling rapidly on every dimension. New users were piling into Facebook. Existing users were increasing their interactions and developing new patterns of usage. The Facebook application was rolling out new features quickly, adding them into the Facebook PHP codebase. A common pattern for scaling a large software application is to use a microservices architecture, breaking up the monolithic application into small services which can scale independently. For many applications, this pattern works well. But for some applications, microservices makes less sense. Microsoft Excel is one example. In Excel, a user is making updates to a complex data model using formulas, functions, and other in-app tools that need to be fast, performant, and integrated. The user needs to have a sense that the Excel data model will update quickly in response to changes. A software team working on a spreadsheet product such as Excel might prefer to keep all the application logic in a monolithic application. A monolith can centralize logic and make it easier to reason about. A monolith can reduce the number of network hops, cutting down on distributed systems problems. Testing and deploying a monolithic application can be less complex than doing so in a distributed, microservices system. Facebook chose to scale its PHP monolith rather than breaking it up into distributed microservices. Scaling PHP allowed Facebook to continue moving fast without going through a painful refactoring that would have slowed down the entire company. The first effort to scale PHP involved transpiling the entire PHP application into C++. This C++ version of Facebook ran faster and with a lower memory footprint. But C++ required ahead-of-time compilation: the PHP codebase had to be converted to C++ in one synchronous step. The Hip Hop Virtual Machine (HHVM) was started to allow the Facebook PHP monolith to run on the V8 JavaScript virtual machine. The advantage of running PHP on V8 was that it allowed Facebook to transpile PHP to JavaScript, an interpreted language that fit better with Facebook’s dynamic application build path. HHVM also allows Facebook to take advantage of the rapidly improving JavaScript runtime environment. Keith Adams was an engineer at Facebook for six years, where he helped develop infrastructure to scale PHP effectively. Keith is now the chief architect at Slack, which is also a scaled PHP application. Keith returns to Software Engineering Daily to discuss why and how Facebook scaled PHP.

Jul 15, 201956 min

Ep 1184Meltano: Data Engineering Lifecycle with Danielle Morrill

Data engineering allows a company to take advantage of the large quantities of data that the company has generated. In many companies, new data has been produced rapidly for many years, but the company has not been able to take full advantage of it. Creating large data sets does not provide immediate value for a company. A company needs to perform data engineering and data science to take full advantage of it. When data gets generated, it is stored in a database, data lake, or API backend like Google Analytics. In order to manipulate that data, it is often pulled into a data warehouse. A data warehouse provides fast access time to large quantities of data. Pulling data from a source like a database or data lake into a data warehouse requires a process known as extract and load. Once the data is in the data warehouse, it may also undergo a transform, which enriches the data or puts it in a format that is easier to make use of. Once data is in a data warehouse, it can be used to build models, interactive dashboards, and Jupyter Notebooks. The data engineering lifecycle has many different components, which is why data engineering can often be intimidating to a company that is trying to make use of their data. Meltano is a project with the goal of providing a system of conventions for managing the data engineering lifecycle. Meltano was started by GitLab, and the Meltano project has some strategic similarities to GitLab. Danielle Morill is the general manager of Meltano at GitLab. She joins the show to discuss the world of data engineering, and the architecture of Meltano. We touch on the different components of a data engineering pipeline, and the most acute pain points for data engineers.

Jul 12, 201956 min

Ep 1183Software Media with Charles Max Wood

Software engineers have a wide variety of media to choose from, including podcasts, blogs, YouTube videos, conferences. The amount of software engineering media that is available is growing and accelerating. Eight years ago, there were not as many options for information about software. Charles Max Wood founded Devchat.tv to create a network of podcasts and other content for software engineers. Today, his podcasts include the popular shows JavaScript Jabber, Ruby Rogues, and Adventures in Angular. Chuck joins the show for a conversation about software media. This was his second time on the show, with his first episode exploring his podcast JavaScript Jabber.

Jul 11, 201957 min

Ep 1182gVisor Container Isolation with Michael Pratt and Yoshi Tamura

Software applications running within a host operating system need to be isolated. Isolation prevents security vulnerabilities, such as one application accessing the memory of another. In modern cloud environments, a single physical host might be running multiple virtual machines on top of a hypervisor. Those virtual machines might be divided up into containers. The different virtual machines and containers might be operated by different users, or even different companies. gVisor is a container sandbox runtime open sourced by Google. gVisor runs containers in a new user-space kernel, and provides a container security system with low overhead. gVisor improves on the previous security properties of containers. Michael Pratt and Yoshi Tamura work on gVisor at Google, and they join the show to talk through the purpose of gVisor and the engineering around the project.

Jul 10, 201952 min

Ep 1181Software Free Radicals with Lior Kanfi

Creating a software company has never been easier. Software engineers are increasingly seeing entrepreneurship as a viable career path. But the path to being an independent software developer is not always clear. Most engineers spend some of their career working at a software company. Even an engineer who intends to build a company someday can thrive within the right environment. Lior Kanfi is a software engineer and the founder of Tikal, a company he started at the height of the dot com bubble in 1999. His initial vision for the company was to build a product around managing knowledge and people within a company. When the dot com bubble burst in the year 2000, it became much harder to run a product-focused business. Companies were not buying lots of experimental software, and investors were not aggressively funding new software companies. After the market collapsed, Lior shifted Tikal from a product-focused company into consulting, in order to have a more reliable income stream. Today, Tikal is a successful software consultancy based in Israel and the Bay Area. Lior describes the engineers within Tikal as “free radical” software developers: independent people who want to learn about new technologies and build experience interacting with clients. Lior joins the show to talk about his 20 year journey building Tikal and the differences between engineers in the Bay Area and those in Israel. Lior also hosted the Full Stack Tech Radar Day in Tel Aviv, which was a great conference I attended.

Jul 9, 201950 min

Ep 1180Edge Storage with Steve Klabnik

Edge computing allows for faster data access and computation. When your client application makes a request, that request might be routed to the edge. Edge servers are more numerous and more widely distributed than normal data centers, but an edge server might not have all of the data or the complete application logic for the backend to serve your request. Edge servers have historically been used for content delivery networks (CDN). CDNs are useful for hosting and serving media files that might otherwise be slow to access over a network. More recently, applications are also using edge servers for computation, as well as storage of resources which are smaller than the movies, music, and images that have traditionally been stored on an edge server. Steve Klabnik is an engineer with Cloudflare, and he returns to the show to discuss storage at the edge. In Steve’s previous appearances we have explored Rust and WebAssembly, and we also touch on those topics in today’s episode.

Jul 8, 201951 min

Ep 1179You Are Not A Commodity (Keynote at Tikal Full Stack Tech Radar Day)

Today’s episode is a keynote I gave at Full Stack Tech Radar Day in Tel Aviv. The talk is called “You Are Not a Commodity”. This talk is also available as a YouTube video. The slides can be accessed here. The world of commodity engineering is coming to an end. Developers are becoming more productive, more flexible, and more entrepreneurial. How does this impact you as an engineer? And how does it impact large companies who need a fungible set of engineers to maintain their software?

Jul 7, 201929 min

Ep 1178ListenNotes: Podcast Search Engine with Wenbin Fang

ListenNotes allows users to find podcasts by categories, popularity, and search queries. ListenNotes is not a podcast client that you download onto your phone, but it allows users to build playlists. These playlists are automatically published to an RSS feed, so that users can curate a playlist on desktop and subscribe to that playlist from any client. Podcasts are growing in popularity. There are more podcasts than ever before, and a podcast search engine needs to refresh its index regularly enough to capture the updates to the podcast universe. A podcast search engine needs to decide what to index. There are many potential fields to choose from within a podcast: the podcast title, the description, the audio transcription, and other metadata–should all of these factor into the results of a search engine? Wenbin Fang is the founder of ListenNotes, and the engineer who has built most of the application. He joins the show to talk about the world of podcasting and his work on ListenNotes, including the business model. Today, ListenNotes makes its money from advertisements and API requests–there is a growing market for applications that want an API for querying the ListenNotes backend.

Jul 5, 201959 min

Ep 1177Permissionless Innovation with Joseph Jacks

Open source software allows developers to take code from the Internet and modify it for their own use. Open source has allowed innovation to occur on a massive scale. Today, open source software powers our consumer client applications and our backend cloud server infrastructure. Linux powers single node operating systems and Kubernetes is the foundation for new distributed systems. Hadoop created an open source distributed file system, Spark gave us a computational runtime on top of it, and Kafka created a middleware platform for shuttling data from one place to another. There are numerous other examples of how open source has changed the world of software development. Open source has also reshaped the business landscape of infrastructure software companies. A common business structure for a modern infrastructure company is the “open core” model. An open core company maintains an open source project that is free to use, but also sells a product or service around that product. Companies with an open core model include Red Hat, HashiCorp, and GitLab. Many companies are building a thriving business with the open core business model. But these companies do not directly control the most important part of the infrastructure supply chain: the cloud provider. Cloud providers have a fundamental tension with open core companies because the cloud providers offer services that compete with the open core companies. In addition to the issue of cloud providers competing directly with the open core companies, some people have questioned whether Amazon Web Services is capturing an unfair portion of the value that is being created by open source. Amazon Web Services is the biggest cloud provider, and it has built a large catalog of services that are built off of open source software. But AWS has not historically contributed heavily to open source relative to the value it has captured. One example of an open core company which has lost market share to an AWS cloud-hosted offering is Elastic, the open core company which maintains the ElasticSearch open source project. Amazon ElasticSearch Service is a closed-source hosted offering built on top of the ElasticSearch. Elastic (the company) has increasingly intermingled proprietary software with their open source repository, making it less clear how that open source repository can be used by companies that want to deploy it for their commercial use. Open core companies such as MongoDB, Redis Labs, and Cockroach Labs have responded to the competitive pressures of AWS by changing their licenses and making it more expensive for cloud providers to offer a cloud-hosted offering of their open source project. The dynamics between cloud providers and open core companies will continue to evolve in the coming years. The norms around open source are up for debate. Joseph Jacks is the founder of OSS Capital, a venture firm focused on investments in commercial open source software companies. He returns to the show to discuss the changing landscape of open core companies, and the benefits of permissionless innovation.

Jul 3, 20191h 0m

Ep 1176Google Earth WebAssembly with Jordon Mears

Google Earth allows users to explore the imagery of the real world. Imagery for Google Earth is taken from satellites, cars equipped with cameras, and other sources. Google Earth renders a data intensive 3-D model of the world on a client application such as a desktop browser or virtual reality system. WebAssembly is a runtime for executing code other than JavaScript in a browser-based environment. WebAssembly is useful for data-intensive workloads, and developers can use programming languages such as Rust or C++ in the browser by compiling to WebAssembly. Jordon Mears works on Google Earth, and he joins the show to talk about the engineering behind Google Earth and how WebAssembly is being used to improve efficiency. He also discusses the state of tooling around WebAssembly today.

Jul 2, 201954 min

Ep 1175FoundationDB with Ryan Worl

FoundationDB is a multi-model distributed key-value store. It is fully ACID compliant and horizontally scalable. FoundationDB is not usually used directly by an application developer–FoundationDB is a foundational building block for higher level distributed systems such as the metadata store for data warehousing tool Snowflake. Ryan Worl is a software engineer who specializes in FoundationDB. He joins the show to discuss the architecture of FoundationDB, including the roles of different server components and the read and write path of FoundationDB. We also talk about applications of FoundationDB, and how it compares to storage engines such as RocksDB and databases such as CockroachDB and Spanner.

Jul 1, 201951 min

Ep 1174FindCollabs Hackathon Winners: Kitspace and Rivaly

FindCollabs is a platform for finding collaborators and building projects. Three months ago we had our first hackathon, with lots of projects being created and collaborated on. In an earlier episode, we showcased the first place winner ARhythm. Today’s show features two more interviews with winners from the first FindCollabs hackathon. Kitspace is an open source hardware registry, and Rivaly is an app for informal ranked leagues, such as ping pong games at work, or board game clubs among your friends. The second FindCollabs hackathon is going on today. Check it out by going to findcollabs.com/open.

Jun 29, 20191h 3m

Ep 1173Citus Data: Founding to Acquisition with Umur Cubukcu

A new database company needs to solve numerous problems in order to succeed. There are already lots of existing database companies, so a new company needs to find a way to strongly differentiate itself. Databases are core infrastructure, so a new database company must earn trust with its customers. A database is a complicated distributed system, so a database company must have strong engineering to get its product to market. Citus Data was founded in 2011. Around that time, companies were looking for solutions to both their online transactional workloads and their large-scale offline analytics workloads. Citus Data built its company around Postgres, a popular database that has been around since 1996. Products from Citus Data include a Postgres scalability extension as well as a cloud-hosted offering for spinning up Postgres instances. Citus Data was acquired by Microsoft earlier this year. In today’s episode, Umur Cubukcu joins the show to talk about the story of the company from its early days to its eventual acquisition. Umur describes the landscape of data systems in 2011 when the company started, and explains how that evolved to the current ecosystem. Umur also talks about how to make an acquisition successful and gives some perspective on the future of data platforms.

Jun 28, 20191h 5m

Ep 1172Lambda School Education with Andrew Madsen

Programmers are in high demand, and software engineering is a career path that is fun, creative, and lucrative. There are many people who want to transition into a career in software and are looking for the right path toward writing code. The traditional college computer science curriculum teaches some software engineering skills, but the time and financial cost of attending a university is prohibitive for many people who want to learn to code. Over the past decade, there have been several new models for software education. Online video platforms such as Udacity and Coursera put computer science courses online to be watched at the viewer’s convenience. Online schools such as Free Code Camp allow someone to learn how to program without any experience, and with no financial payment. Boot camps with income-sharing agreements such as App Academy create an in-person education environment that mimics a university, but with better cost structure and incentive alignment. Lambda School is an education system that takes elements of other software education models and combines them with newer SaaS technologies such as Slack and Zoom videoconferencing. Lambda School is an online software engineering curriculum with an income sharing agreement. “Income sharing agreement” means that the student does not pay for their education until they get a job. With this model, the student can pay back Lambda School after their software engineering education gets them a high-paying software engineering job. Andrew Madsen works at Lambda School, and he joins the show to describe the path that a student takes through Lambda School, the school’s curriculum for software education, and how Lambda School differs from the other options for coding education.

Jun 27, 201958 min

Ep 1171Afresh: Grocery Store Software with Volodymyr Kuleshov

A grocery store contains fruit, vegetables, meat, bread, and other items that can expire. In order to keep these items in stock, the store must be aware of how much food has been sold and what has gone bad. When a food item is low in stock, the store needs to order more of that food from a central distribution system. Managing food inventory is not simple. Some kinds of meat might expire faster than others. Avocados do not become ripe at the same rate as apples. In order to keep the shelves stocked, there are manual workflows for checking the inventory and ordering new inventory. Afresh is a company that builds software for grocery stores. Afresh works with grocery chains that have a central distribution center. These grocery stores already have some software. At the back of the store, inventory management systems maintain records of the items that the store has on the shelves. At the front of the store, checkout systems detect what has been sold and help to update inventory. When the inventory is running low, the store can order more inventory from the central distribution center, so that trucks can deliver more inventory. Afresh improves the operational intelligence of these stores by detecting spoilage among items that are prone to expiration, such as fruit. Volodomyr Kuleshov is the CTO and co-founder of Afresh and he joins the show to discuss the technical challenges of a grocery store, and the software that Afresh is building to make groceries more intelligent.

Jun 26, 201948 min

Ep 1170Airflow in Practice with Chaim Turkel

Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines. Chaim Turkel is a backend data architect at Tikal. He joins the show to discuss a case study of using Airflow to rearchitect the data engineering workflow of a complex financial application. We discussed the problems that Airflow solves and the process of porting existing workflows to Airflow.

Jun 25, 201953 min

Ep 1169Project Management with Kurt Schrader

Software projects are organized and planned using project management software. Examples of project management software include JIRA, Trello, and Asana. There are hundreds of tools for managing a software project because there are infinite ways that a project could be managed. Google Docs changed project management by allowing documents to be easier to share and collaborate on. Newer SaaS tools such as Slack, Dropbox, and Notion have taken the design lessons from social networking apps to make enterprise software more engaging. As the tools improve, our project management strategies change, and new software tools emerge to fit those new management strategies. Kurt Schrader is the CEO of Clubhouse, a project management tool for software engineers. Kurt joins the show to talk about the history and future of project management tools. He also discusses the engineering challenges of improving performance on a complicated webapp. Project management tools often have to load hundreds of small objects on a page, which required performance optimizations in the Clubhouse frontend JavaScript library.

Jun 24, 201957 min

Ep 1168Niantic Real World with Paul Franceus

Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile technology makes it possible for a smartphone to display rendered 3-D images over a video stream without running out of battery. Ingress was the first game to come out of Niantic, followed by Pokemon Go, but there are other games on the way. Niantic is also working on the Niantic Real World platform, a “planet-scale” AR platform that will allow independent developers to build multiplayer augmented reality experiences that are as dynamic and entertaining as Pokemon Go. Paul Franceus is an engineer at Niantic, and he joins the show to describe his experience building and launching Pokemon Go, as well as abstracting the technology from Pokemon Go and opening up the Niantic Real World platform to developers. Niantic developer contest finalists

Jun 21, 201954 min

Ep 1167WebAssembly Compilation with Till Schneidereit

WebAssembly allows for web-based execution of languages other than JavaScript. Programs written in Rust or C++ can be compiled down to WebAssembly and shipped over the browser for on-the-fly execution in a safe, memory controlled environment. WebAssembly has been in development for more than two years, and is still an immature ecosystem because building the necessary tooling for WebAssembly is hard. Much of the web has been built around JavaScript and the V8 JavaScript engine, which has been tuned to optimize an interpreted language (JavaScript). WebAssembly modules are often written in C++ or Rust, which are compiled languages. There are engineering challenges at the edge between the interpreted JavaScript runtime and the precompiled WebAssembly modules. Till Schneidereit is a senior research engineering manager at Mozilla. He joins the show to discuss the compilation path of WebAssembly and the state of the ecosystem.

Jun 20, 201952 min

Ep 1166Infrastructure Wars with Sheng Liang

Sheng Liang was the lead developer on the original Java Virtual Machine. Today he works as the CEO of Rancher Labs, a company building a platform on top of Kubernetes. Sheng joins the show to discuss his experiences in the technology industry. The container orchestration wars had many victims. The competing standards for how an enterprise should manage its numerous containers caused several companies to go down a path where they were building infrastructure which eventually had to be replaced. As Sheng discusses in today’s episode, the container orchestration wars almost killed his company. Rancher was originally built on top of a different container orchestrator, and the migration to Kubernetes required a massive rewrite of the Rancher platform. The container orchestration wars were not the first technology battle that Sheng has seen in his career–and it won’t be his last. In today’s show, we discuss the nature of technology wars. Are they necessary? How can a software company minimize the damage caused by a war between competing standards? Sheng was an excellent guest and we didn’t cover nearly as many subjects as I wanted to, so we will have to do another show in the future!

Jun 19, 201956 min

Ep 1165Kubernetes Operators with Rob Szumski

Kubernetes has made distributed systems easier to deploy and manage. As Kubernetes has become reliable, engineers have started to look for higher level abstractions we can define on top of Kubernetes. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are useful for spinning up distributed systems such as Kafka, Redis, or MongoDB. These data systems are complicated, stateful applications with lots of failure domains. The operator framework enables a developer to deploy one of these complicated applications with less fear of the system crashing, or entering an erroneous state. Rob Szumski is an engineer at Red Hat. He joins the show to discuss Kubernetes, the operator pattern, and his time at CoreOS, which was acquired by Red Hat.

Jun 18, 20191h 5m

Ep 1164Render: High Level Cloud with Anurag Goel

Cloud computing was popularized in 2006 with the launch of Amazon Web Services. AWS allowed developers to use remote server infrastructure with a simple set of APIs. But even with AWS, it was still not simple to deploy and manage a web application. In 2007, Heroku launched a platform built on top of AWS. Heroku focused on the developer experience by optimizing for users who were deploying Ruby on Rails applications. Since then, Heroku has expanded into other forms of managed infrastructure, including other application frameworks like NodeJS, and databases like Postgres. Heroku was the first popular “layer 2” cloud provider. Twelve years later, it is probably still the most popular. But there have been many other cloud providers built on top of AWS, including Netlify, Zeit, Spotinst, and Firebase. Layer 1 cloud providers are Google Cloud, AWS, Azure, Digital Ocean, and other raw infrastructure providers. These companies provide a great service in their low cost, commodity infrastructure. But the layer 1 providers are not optimizing for developer experience. They need to cater to a broad set of developers, some of whom want to work at a low level. A layer 2 cloud provider can build an opinionated solution that serves a subset of the overall cloud market particularly well. Render is a layer 2 cloud provider that optimizes for specific developer workflows, such as deploying a NodeJS web server, a static site, or a Docker container. Anurag Goel is the founder of Render, and he joins the show to discuss the strategy and the economics of Render. Anurag was also one of the early employees at Stripe, and he discusses his experience and learnings from working at the company.

Jun 17, 20191h 16m

Ep 1163Elegant Puzzle with Will Larson

Software engineering is an art and a science. To manage engineers is to manage artists and scientists. Software companies build practical tools like payment systems, messaging products, and search engines. Software tools are the underpinnings of our modern lives. You might expect this core infrastructure which modern humans rely on to have been constructed with pure formulaic rigor. But the best software tools are not built within a totally defined process. Software is built through messy iteration. When a piece of software looks pristine, that is often a function of how many mistakes have been made, and then subsequently corrected for. There is no fixed process for how to build good software. As our tools get better, we have to update our software engineering practices to utilize those new tools. We have to rethink the style that we are working in. We have to discard old tools and procedures in order to pick up the new ones, and have higher leverage. As an organization scales, the structure of the organization needs to be modified. Team members need to be reallocated. Checks and balances need to be put in place. Rules and cultural practices need to codified, because a larger organization cannot have ties broken by an individual. Software is built by humans, and every management decision must be considered in the light of human psychology. When we change a line of code, the code does not get emotional about being altered. But the same cannot be said of humans. Even a minor conversation between an engineering manager and a direct report can have lasting implications. Will Larson is the author of An Elegant Puzzle: Systems of Engineering Management. He works on Foundation Engineering at Stripe, and has worked in engineering management at Uber, Digg, and other software companies. Elegant Puzzle provides strategies, tactics, and ruminations on software development. Will joins the show to explore the multifaceted subject of engineering management.

Jun 14, 20191h 7m

Ep 1162Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire

Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies want to make the tools more accessible to the developers across the organization. There are many steps that an engineer must go through to use machine learning, and each additional step inhibits the chances that the engineer will actually get their model into production. An engineer who wants to build machine learning into their application needs access to data sets. They need to join those data sets, and load them into a machine (or multiple machines) where their model can be trained. Once the model is trained, the model needs to test on additional data to ensure quality. If the initial model quality is insufficient, the engineer might need to tweak the training parameters. Once a model is accurate enough, the engineer needs to deploy that model. After deployment, the model might need to be updated with new data later on. If the model is processing sensitive or financially relevant data, a provenance process might be necessary to allow for an audit trail of decisions that have been made by the model. Rob Story and Kelley Rivoire are engineers working on machine learning infrastructure at Stripe. After recognizing the difficulties that engineers faced in creating and deploying machine learning models, Stripe engineers built out Railyard, an API for machine learning workloads within the company. Rob and Kelley join the show to discuss data engineering and machine learning at Stripe, and their work on Railyard.

Jun 13, 20191h 5m

Ep 1161Software Businesses with Hiten Shah

The software market changes every year. As individuals and enterprises become more willing to buy software, there are new markets for entrepreneurs to sell software to. Good software has high margins and high retention, so even a niche software business can prove profitable. As software spreads across the world, developing countries are showing a willingness to buy the same software tools as the developed countries, making the niche software businesses even more successful than expected. It’s a great time to start a software business. But the fundamental challenges faced by an entrepreneur have not gone away. An entrepreneur must find the necessary capital, build an initial product, iterate on that product, market to their audience, sell to their customers, and retain their users. Hiten Shah is an entrepreneur and investor who has started several businesses, both successful and unsuccessful. He is also the host of the Startup Chat podcast, a show about strategies and tactics for software businesses. Hiten joins the show to discuss his experience in the software industry, including lessons on financing, product development, and the future of software products.

Jun 12, 201955 min

Ep 1160Kubernetes Vision with Joe Beda

Google Cloud was started with a vision of providing Google infrastructure to the masses. In 2008, it was not obvious that Google should become a cloud provider. Amazon Web Services was finding success among startups who needed on-demand infrastructure, but the traditional enterprise market was not yet ready to buy cloud resources. Googlers liked the idea of becoming a cloud provider. But was it the right time to enter the market? Google’s advertising business was a large and growing cash cow. Executives within Google were not sure how much capital and effort should be allocated into an infrastructure business. When Google decided to go into the cloud business, Joe Beda was one of the engineers who helped lead the effort, and joins the show as today’s guest. Google’s internal server infrastructure is managed by Borg, a system for allocating resources to applications. Google Cloud runs on Borg, and there were a number of early engineering challenges to building the necessary functionality into Borg for running a cloud provider on top of it. One example of a technical challenge that Google faced was the refactoring of Borg to run Google Cloud workloads. The requirements for public infrastructure are different than those of internal Googlers. Inside of Google, developers deploy their applications to containers running on bare metal. Outside of Google, developers want to create virtual machines. Borg needed to be refactored in order to instantiate VMs. Google solved this technical problem, as well as many other challenges, and Google Cloud slowly gained momentum in the market. But AWS remained the default choice for profitable enterprise workloads. It wasn’t until the container orchestration wars that Google found an opportunity to jump on a market segment that offered strong differentiation. By open sourcing Kubernetes and presenting a clear vision for where the project was going, Google shifted the battlefield of the public cloud toward a competitive landscape where it has many advantages. Kubernetes also provided many other technology companies with an opportunity to get into the cloud market, creating a collaborative, multi-company ecosystem that has accelerated the pace of software faster than anyone expected. Joe Beda has been instrumental in the evolution of the cloud native ecosystem. In today’s episode, Joe gives his memories on Google Cloud, Kubernetes, and his Kubernetes company Heptio, which he sold to VMware.

Jun 11, 20191h 7m

Ep 1159Kubernetes Storage with Saad Ali

Containers are made to fail gracefully. When your container shuts down due to a hardware or software failure, your distributed application should be able to tolerate that failure. One simple way to be able to tolerate such a failure is to make all of your application logic “stateless.” If your application does not maintain state, then shutting it down in the middle of a computation is not a problem–you can just restart the application, restart your computation, and get the same result. But applications need to maintain state. We need to use databases and in-memory systems to manage long-lived user sessions and other interactions. A database is not just an on-disk abstraction–a database requires an application server to be accepting network traffic. We can run those database applications within containers. There is a fundamental tension between stateful applications and the idea that containers are meant to tolerate failure gracefully. Saad Ali is an engineer at Google, and he returns to the show to discuss Kubernetes storage and state management. He gave a keynote at KubeCon EU, which I spoke to him about. New Software Daily app for iOS. You can become a paid subscriber for ad free episodes at softwareengineeringdaily.com/subscribe

Jun 10, 201955 min

Ep 1158Kubernetes Market with Adam Glick

Amazon Web Services is the leading cloud provider by a large margin. Amazon established its lead by being first to market in 2006, with Google and Microsoft taking several years to catch up to the huge business opportunity of the cloud. Since 2008, Google Cloud has been working on cloud products for developers. It started with App Engine, which is widely used internally at Google, but has not had overwhelming public adoption. Over the last eleven years, Google has refined its understanding of how customers want to buy public cloud resources. Google Cloud products like Cloud Storage, Persistent Disk, and BigTable have given Google parity with many of the AWS public cloud offerings. Although Google has caught up to AWS in terms of products, the enterprise market has continued to choose AWS as its default. AWS is widely perceived as having more experience in running enterprise workloads, and a better responsiveness to customers. In order to keep Amazon from running away with the cloud market entirely, Google needed to shift the competitive landscape to different territory. Kubernetes provided the paradigm shift that Google needed. The market for cloud providers has changed completely due to Kubernetes. When Google open sourced Kubernetes, it created a common codebase for software companies to build software for managing distributed systems. In the span of five years, Kubernetes has turned the world of cloud products into a world resembling the open source Linux ecosystem. This is a remarkable shift, and every infrastructure software vendor is still figuring out its strategy for adapting. Adam Glick is the head of modern infrastructure and serverless marketing at Google. With Craig Box, he hosts the Kubernetes Podcast from Google, an excellent show about recent developments and evergreen concepts within the world of Kubernetes. Prior to Google, Adam worked at AWS for 3 years and Microsoft for twelve years. He has seen each of the major cloud providers up close and has a deep awareness for how each company thinks. We had a great conversation about the cloud native landscape, podcasting, and developer marketing. New Software Daily app for iOS. You can become a paid subscriber for ad free episodes at softwareengineeringdaily.com/subscribe

Jun 7, 20191h 14m

Ep 1157Service Mesh Interface with Lachlan Evenson

Containers offer a lightweight abstraction for running a server. Cloud providers are able to manage billions of containers from different users, allowing for economies of scale so that each user can pay less. Today, there is a variety of ways that users can deploy containers on a cloud provider. These containers can run in managed Kubernetes clusters, in functions-as-a-service, or in long-lived standalone container instances. User preferences are getting more sophisticated, with some users showing an interest in Knative, an open source serverless system originally created at Google. Whichever container deployment system you choose, your application and its multiple servers need a way to route traffic, measure telemetry, and configure security policy. A service mesh abstraction can help serve these use cases. Lachlan Evenson has worked in containers and Kubernetes since before the container orchestration wars. He was an engineer at Deis, a company which built an open source platform-as-a-service running on top of containers and Kubernetes. Deis was acquired by Microsoft, where Lachlan now works as principal program manager of Container Compute. Lachlan joins the show to discuss containers, Kubernetes, and the Service Mesh Interface, an interoperable service mesh layer that Microsoft launched with Buoyant.

Jun 6, 20191h 0m

Ep 1156Multicloud Future with Bassam Tabbara

Each cloud provider offers a different set of services which are not always compatible with each other. What are the challenges of building an application that interoperates with multiple different clouds? The first issue is API compatibility. Most cloud providers have a managed SQL offering, a bucket storage system, and server abstractions like virtual machines and containers. But these tools might have different APIs on each cloud. The code that you wrote to save your application data to Amazon S3 might have to be rewritten if you decide to switch your bucket storage to a different provider. Another issue that interferes with cloud interoperability is the degree of integration on a particular cloud. If I build my application for AWS, I might be heavily integrated with Amazon’s Identity and Access Management policy system and AWS logging. Each cloud provider makes it particularly easy to connect to their integrated solutions. There is also the problem of services on one cloud that simply do not map to a service on any other cloud. Google Cloud Bigtable does not have an equivalent service on Amazon. Microsoft CosmosDB does not have an equivalent service on Digital Ocean. As developers, this irritates us. We want to be able to deploy our application to any cloud. We want to be able to move applications easily from one cloud to another. And we want to mix tools from different clouds together as easily as we import a library. Bassam Tabbara is the CEO of Upbound, a company focused on making multicloud applications easier to deploy and operate. I spoke to Bassam at KubeCon EU 2019, and he described the problems of multicloud deployments and the opportunities for the cloud native ecosystem to become more cross-compatible.

Jun 5, 20191h 6m

Ep 1155Kubernetes Development with Tim Hockin

Kubernetes has evolved from a nascent project within Google to a thriving ecosystem of cloud providers, open source projects, and engineers. Tim Hockin is a principal software engineer who has been with Google for 15 years. Tim joins the show to talk about the early days of the Kubernetes projects, and the engineering efforts that are under way five years into the project. At KubeCon EU 2019, two of the prevalent subjects of discussion were service mesh and serverless–particularly the Knative project. Tim gave his perspective for how projects that are adjacent to Kubernetes are developed within the community.

Jun 4, 201946 min

Ep 1154Google Anthos with Aparna Sinha

Google’s cloud business was long regarded as a place where startups could build a business, but not established enterprises. For serious workloads, enterprises chose Amazon almost unanimously. This phenomenon of Amazon as the default was described by a phrase that harkened back to the days of IBM’s dominance: “nobody ever got fired for choosing AWS.” With the rise of Kubernetes, Google has established a reputation as a reliable provider of container orchestration. As enterprises look to roll out Kubernetes workloads, Google is convincing many of them to work with Google Kubernetes Engine. Kubernetes is one part of the changes which fall under the description of “digital transformation”. As these enterprises build out their digital transformation strategy, they are also looking for a strategy around multicloud, on-prem, policy management, and consulting partnerships. Anthos is a platform where enterprises manage the resources and configuration of their cloud deployments, as well as a system to partner with services integrators and independent software vendors. Aparna Sinha works on Anthos, Google Kubernetes Engine, and other projects at Google. In today’s show we discuss the process of digital transformation, as well as the tactics for how a digital transformation is actually implemented. We also talked about GKE on-prem, and the kinds of tooling needed by on-prem application deployments.

Jun 3, 201952 min

Ep 1153Service Mesh Wars with William Morgan

The CNCF was formed under the looming shadow of AWS. The CNCF was seeded with the donation of Kubernetes by Google. Much like the Linux community was positioned as a rebellious movement in reaction to Microsoft’s dominance, the Kubernetes community represents a fervent desire to open up the market to cloud providers beyond the tight-lipped, proprietary dominion of Amazon. With such a deep spirit of insubordination, it is no surprise that the community has rejected Istio like a set of loosely coupled organs rejecting a foreign skin attempting to layer itself across them. Even though the CNCF was founded by Google, the community was formed in spite of big centralized clouds, not as a marketing vessel for their products which may or may not be open source. Microsoft seems to understand this fact better than Google, at least in the domain of service mesh. The day after this interview with William, Microsoft announced the Service Mesh Interface (SMI), a project it partnered with Buoyant and other companies on to create a minimal spec for what a service mesh should offer to a Kubernetes deployment. The SMI presents a safe buy-in point for enterprises who want a service mesh, but do not want to get caught in the evangelistic crossfire of Istio and Linkerd. It is in this environment that we begin our next series of shows on the current cloud native ecosystem. Thanks to the Cloud Native Computing Foundation for putting together an amazing podcasting zone at KubeCon, and allowing me to conduct these interviews.

May 31, 20191h 21m

Ep 1151Digital Transformation: Capital One with Hillary McTigue

Cloud computing, open source, and mobile computing are trends that affect every organization. When a large organization adapts to these trends, it is commonly referred to as a “digital transformation”. Digital transformation causes many companies to reframe their business as a software company. A candy manufacturer now must think of itself as a software company that makes candy. An insurance company must now think of itself as a software company that issues insurance. Capital One is a bank that was started in 1988. Capital One has always had an emphasis on software, but the company’s digital transformation has affected it as much any other company. The company is migrating to the cloud, building out microservices, rolling out continuous delivery pipelines, and shifting the internal culture to be more adept at using software. Hillary McTigue is a senior director of data engineering at Capital One. She joins the show to discuss her experience implementing a digital transformation within a large company. Subjects we discuss include culture, management strategy, and the sequencing of different phases of a digital transformation. Full disclosure: Capital One is a sponsor of Software Engineering Daily. We are hiring two interns for software engineering and business development! If you are interested in either position, send an email with your resume to [email protected] with “Internship” in the subject line.

May 30, 201950 min

Ep 1150Netflix Early Days with Greg Burrell

Netflix started with a DVD-by-mail product. The software infrastructure and operations practices needed for the DVD business were very different from those needed by a streaming video company. Since the early days of Netflix, CEO Reed Hastings knew that the company would evolve to becoming a streaming video platform. But he did not know when the technology would be advanced enough to support video streaming, and he did not know how users would consume it. Greg Burrell has worked at Netflix for 14 years. Greg was one of the first engineers to start working on video streaming, which Netflix first attempted to implement with a set top box that downloaded movies and played them on your television. After evolving this strategy, Netflix arrived at the current model of video streaming through apps on browsers and mobile devices. As the company pivoted from DVD-by-mail to video streaming, Netflix encountered multiple challenges across engineering, operations, and communications across the company. At the time, there was no “DevOps” movement. There were not established continuous delivery practices. The available cloud technologies were immature and low level. Greg joins the show to describe the evolutionary arc of Netflix’s engineering process. Greg also presents a model for software development that he describes as “Full Cycle Development”. At Netflix, engineering teams of full cycle developers work without dedicated operations or testing teams. It is a sophisticated approach to engineering management. I spoke to Greg at the Fullstack Tech Radar Day, a software conference in Tel-Aviv put on by Tikal, an engineering community based out of Israel and San Francisco. This was a great conference, and we’ll be airing some additional content from it in the coming weeks. We are hiring two interns for software engineering and business development! If you are interested in either position, send an email with your resume to [email protected] with “Internship” in the subject line.

May 29, 20191h 8m

Ep 1149Augmented Reality Gaming with Tony Godar

Augmented reality applications can be used on smartphones and dedicated AR headsets. On smartphones, ARCore (Google) and ARKit (Apple) allow developers to build for the camera on a user’s smartphone. AR headsets such as Microsoft HoloLens and Magic Leap allow for a futuristic augmented reality headset experience. The most prominent use of augmented reality today is gaming, with a notable example being Niantic’s Pokemon Go. Tony Godar is a software engineer who works on augmented and virtual reality applications. He joins the show to talk about his day job working on virtual reality experiences, and an AR game he built called ARhythm. Tony was the winner of the FindCollabs Hackathon and we also discussed his experience working on the project through FindCollabs. RECENT UPDATES: The FindCollabs Open has started. It is our second FindCollabs hackathon, and we are giving away $2500 in prizes. The prizes will be awarded in categories such as machine learning, business plan, music, visual art, and JavaScript. If one of those areas sounds interesting to you, check out findcollabs.com/open! The FindCollabs Podcast is out! We are booking sponsorships for Q3, find more details at https://softwareengineeringdaily.com/sponsor/

May 28, 201947 min

Ep 1148CamelCamelCamel: Amazon Price Tracker with Daniel Green

CamelCamelCamel is a tool for tracking prices on items on Amazon.com. The company was launched eleven years ago and is built off of the Amazon Product Advertising API. Daniel Green is one of the founders of CamelCamelCamel, and he joins the show to describe his experience building the product. Amazon and CamelCamelCamel (C3) have a complex relationship. C3 makes its money from referral listings. When a user tracks the price of an Amazon item using C3, that user will probably eventually click on the referral listing on C3. If the user purchases the item on Amazon, C3 gets a percentage of the purchase from Amazon. C3 has millions of product listings where they are tracking the price of items on Amazon. They have created a directory with a large subset of Amazon’s items by leveraging an API that was originally meant for advertising. Whether or not this is a proper use of the API, C3 arguably leads to more purchasing volume on Amazon–which is ostensibly why Amazon lets the company continue to operate. In today’s show, Daniel gives a history of CamelCamelCamel, including his own background as an engineer working in finance and how he wound up working on C3 for the past decade. We also talk through how C3 is architected. RECENT UPDATES: The FindCollabs Open has started. It is our second FindCollabs hackathon, and we are giving away $2500 in prizes. The prizes will be awarded in categories such as machine learning, business plan, music, visual art, and JavaScript. If one of those areas sounds interesting to you, check out findcollabs.com/open! The FindCollabs Podcast is out! We are booking sponsorships for Q3, find more details at https://softwareengineeringdaily.com/sponsor/

May 24, 201958 min

Ep 1147Gaming with Eli Brown

Gaming is becoming mainstream. Popular multiplayer games such as Fortnite and Minecraft present players with a massive virtual world to explore, build, and compete within. Turn-based games such as Hearthstone and Magic are breeding a new generation of board game and card game aficionados. Social media networks like Twitch and YouTube have turned gaming into a voyeuristic sport that is outcompeting many physical sports games for attention. Guilded is a platform for managing gaming teams. On Guilded, there are teams for games like League of Legends, Fortnite, and World of Warcraft. These teams use Guilded to manage calendars, Discord bots, forum software, documents, statistics, and recruiting. This might sound confusing–why does a gaming team need document management, calendars, and analytics? Are we talking about a video game team or a software company? To understand Guilded, you need to understand the rapidly changing modern gaming ecosystem. Eli Brown is a founder of Guilded.gg. He joins the show to talk about the world of gaming, its intersection with social media, and the fascinating engineering problems involved in building a platform for gaming teams. RECENT UPDATES: The FindCollabs Open has started. It is our second FindCollabs hackathon, and we are giving away $2500 in prizes. The prizes will be awarded in categories such as machine learning, business plan, music, visual art, and JavaScript. If one of those areas sounds interesting to you, check out findcollabs.com/open! The FindCollabs Podcast is out! We are booking sponsorships for Q3, find more details at https://softwareengineeringdaily.com/sponsor/

May 23, 201952 min