Software Engineering Daily

Building and nurturing a developer community is hard work but it is vital for the growth of a country’s technology ecosystem. When communities coalesce around programming languages, tools or programming methods, what follows is a network of conferences, meet ups and other similar events. Juan Pablo Buriticá, VP of Engineering at Splice, has spent the last decade building developer communities in his home country of Colombia and across Latin America as well as running distributed engineering teams. He has helped to launch a number of conferences, meet-ups and more recently, an online meet-up providing advanced technical information for native Spanish speakers. In today’s episode, Juan returns to Software Engineering Daily to speak with Carl Mungazi about the benefits of having a distributed engineering team and hiring developers from developing nations. He discusses how Colombia came to have the largest Spanish-speaking JavaScript community in the world and the importance of good communication when building software with a team of developers based around the world.

Quality assurance testing is a form of testing that closely mirrors user behavior. Sometimes it is manual, sometimes it is automated. Automated QA tests are scripts that validate correct data representation as the application mechanically runs through high-level workflows–like a login page. Manual QA testers act out use cases of an application to see if there are any bugs that were missed during automated test cases. Manual QA testing is often necessary for complex applications where it is not possible to enumerate all potential workflows within a script. Different companies have radically different workflows for QA testing. There are a variety of ticketing systems, testing frameworks, and team chat applications that play a role in a tester’s daily life. QASymphony is a platform for testing tools that integrates with other popular technologies to centralize a QA testing workflow. Jonathan Alexander is the CTO at QASymphony. He’s also the author of Codermetrics: Analytics for Improving Software Teams. He joins the show to discuss the past and present of QA and his strategies for managing the team that is building QASymphony. Thanks to Kevin Wolf for the intro.

Facebook was rapidly outgrowing its infrastructure in 2009. Classic data center design was not up to the task of the rapid influx of new users and data, photos and streaming video hitting Facebook’s servers. A small team of engineers spent the next two years designing a data center from the ground up to be cheaper, more energy efficient, and more ergonomic for the engineers who worked within. That data center design was open sourced in 2011. Intel, Rackspace, and Goldman Sachs were the first three large organizations to join Facebook in the Open Compute Project, an effort to bring the benefits of open source collaboration to data centers. Steve Helvie works on the Open Compute Project and he joins the show to describe how the project has evolved in the last six years–how it has affected data center design and the implications for the future.

Slack is an application for team communication. Users chat across mobile devices, web browsers, and a desktop application, which means Slack has three places to deploy on rather than two. And the desktop apps on Windows, Mac, and Linux are not identical, so Slack has even more places to deploy. With so many different runtime environments, Slack needs to make technology choices that reduce the chance of errors. TypeScript allows for static typing of JavaScript. The extra compilation step checks the types of variables being passed between different places–so the errors will be discovered at compile time. In an untyped world, those errors might occur at runtime. TypeScript also unlocks the ability to put JavaScript code in an IDE, allowing for more efficient development. Felix Rieseberg is a desktop engineer at Slack, and in today’s episode he explains the unique challenges of building Slack, and why the team moved from JavaScript to TypeScript. TypeScript at Slack – engineering blog

Animations make an application more fun and engaging. For most apps, animation is an afterthought. Developers are concerned with getting the functionality right, and designers have enough work to do simply getting icons, text formatting, and page layout correct. There is also the issue of cross-device compatibility. iOS, Android, and web have different ways of doing animation, with no unifying standard–except gifs, and gifs are not interactive, they simply play from start to finish. Airbnb’s emphasis on design makes it the right company to work on the problem of cross-device, interactive animations. Brandon Withrow and Gabriel Peal are engineers who work on Lottie, a library for animations in iOS, Android, web, and React Native. This episode is about how and why Lottie was built, and how Lottie gets used within Airbnb.

JavaScript is moving so fast. It’s not easy to keep up with all of the frameworks, build tools, and packages. No other language spans frontend to backend, mobile to web to server. Sacha Greif is an independent designer and developer most prominent in his roles as co-author of Discover Meteor and community builder at Sidebar.io, a design newsletter with over 35,000 subscribers, and Hacker News Kansai. He is currently best known in the Javascript community as the maintainer of VulcanJS, and for his annual State of Javascript survey which is now open for 2017. In this episode, Shawn Wang guests hosts a discussion about both projects and Sacha’s thoughts on independent web design and development.

The Internet of Things is the concept that traditionally analog objects, like thermostats and lightbulbs, can be given digital guts and connected to the internet to create more value for users. From Nest thermostats to Phillips Hue lightbulbs, these connected things are starting to enter the mainstream. According to recent estimates by Gartner, over eight billion connected “Things” will be in use in 2017, with that number ballooning to over twenty billion by 2020. Jeremy Foster is a Technical Evangelist at Microsoft and the host of Microsoft Virtual Academy’s “Introduction to Azure IoT” course. In this episode, host Jared Porcenaluk joins Jeremy to discuss developing for the Internet of Things.

Serverless computing reduces the cost of using the cloud. Serverless also makes it easy to scale applications. The downside: building serverless apps requires some mindset shift. Serverless functions are deployed to transient units of computation that are spun up on demand. This is in contrast to the typical model of application delivery–the deployment of an application to a server or a container that stays running until you shut it down. Robin Weston develops large projects with AWS Lambda, and he joined me for a discussion of how to build applications for serverless environments and how to do continuous delivery with serverless functions. One big appeal for continuous delivery fans is that serverless deployments are often smaller–the user is deploying something as small as a function. Full disclosure: ThoughtWorks GoCD is a sponsor of Software Engineering Daily. Serverless Architectures and Continuous Delivery by Robin Weston Robin Weston at Pipeline Conf

After raising $18 million, social networking startup Yubl made a series of costly mistakes. Yubl hired an army of expensive contractors to build out its iOS and Android apps. Drama at the executive level hurt morale for the full-time employees. Most problematic, the company was bleeding cash due to a massive over-investment in cloud services. This was the environment in which Yan Cui joined Yubl. The startup did have traction. There were social media stars who would announce on Twitter that they were about to go on Yubl, and Yubl would be hit by an avalanche of traffic. 50,000 users suddenly logging on to interact with their favorite celebrity was a significant traffic spike. How do you deal with a traffic pattern like that? Serverless computing. AWS Lambda allowed the company to scale up quickly in a cost efficient manner. Yan began refactoring the entire backend infrastructure to be more cost efficient, heavily leveraging AWS Lambda. Unfortunately, Yan’s valiant effort was not enough to save the company. But there are some incredible engineering lessons from this episode–how to build cost-effective, scalable infrastructure. It’s also a case study worth looking at if you work at a startup, whether or not you are an engineer. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Quantum computing is based on the system of quantum mechanics. In quantum computing, we perform operations over qubits instead of bits. A qubit is a vector, which can take on many more values than 0 or 1. The technology used to implement quantum computers is advancing such that it has its own Moore’s Law, but it can also leverage the classical advancements of Moore’s Law. If classical computing advances at the exponential pace of 2^n, quantum computing advances at the pace of 2^2^n. Quantum computing will advance technology in ways that will take us by surprise. If things feel like they are moving fast now, just wait until developers have access to quantum processing units. Machine learning, simulated chemical synthesis, and NP-complete problems are ripe for quantum computers. Vijay Pande is a partner at Andreessen Horowitz and a board member at Rigetti Computing, a quantum computer company. In this episode, we explored what software engineers today need to know about quantum computers and some of the application domains that developers will be working on as quantum computers become available. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Continuous delivery is a model for deploying small, frequent changes to an application. In a continuous delivery workflow, code changes that are pushed to a repository set off a build process that spins up a new version of the application. Testing is performed against that new build before advancing it to production, merging it with the existing codebase. Many continuous delivery products are getting built today because it is a wide open space–much like cloud providers or monitoring tools. There are subjective product and engineering decisions to be made depending on the audience for the product. Heroku Flow is a continuous delivery platform built on top of Heroku, a platform as a service. Andy Appleton is an engineer at Heroku and he joins the show to describe how Heroku Flow was built. Two years of work went into the project from initial conception to launch. Full disclosure: Heroku is a sponsor of Software Engineering Daily. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Patents allow individuals and company to lay creative claim for an invention. A patent can provide protection from having its idea being used without giving credit to its creators. Of course, is that patents can be filed and not turned into products, inhibiting innovation. Patents can also be used offensively in a practice known as patent trolling. Large companies like IBM and Google have so many patents that they have trouble keeping track of them all. And if your company has many different hardware and software products, how can you be sure that your patent collection protects you from a patent troll? Nicole Shanahan is the CEO of ClearAccessIP, a product that indexes patents, looks for vulnerabilities in a corporation’s patent strategy, and finds opportunities in a patent collection for further value. The large text corpus of a patent collection is the perfect place to apply machine learning. We discussed the nature of patents, the intersection between law and software, and the product development process of ClearAccessIP. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Wearables are everywhere. In the medical field they are transforming lives. Haiyan Zhang, Innovation Director at Microsoft Research, created a wearable for a young graphic designer that developed Parkinson’s. Parkinson’s is a condition that inhibits movement, and this wearable allows the Parkinson’s patient to write and draw again. Haiyan explained the research process and the technical aspects of how it works. Edaena Salinas of The Women in Tech Show interviewed Haiyan for this episode. They talked about the Internet of Things, the components of these systems and the technical challenges. Haiyan also explained her path from software engineering to design and the process of commercializing products that come from research. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Self-driving cars are here. Fully autonomous systems like Waymo are being piloted in less complex circumstances. Human-in-the-loop systems like Tesla Autopilot navigate drivers when it is safe to do so, and lets the human take control in ambiguous circumstances. Computers are great at memorization, but not yet great at reasoning. We cannot enumerate to a computer every single circumstance that a car might find itself in. The computer needs to perceive its surroundings, plan how to take action, execute control over the situation, and respond to changing circumstances inside and outside of the car. Lex Fridman has worked on autonomous vehicles with companies like Google and Tesla. He recently taught a class on deep learning for semi-autonomous vehicles at MIT, which is freely available online. There was so much ground to cover in this conversation. Most of the conversation was higher level. How do you even approach the problem? What is the hardware and software architecture of a car? I enjoyed talking to Lex, and if you want to hear more from him check out his podcast Take It Uneasy, which is about jiu jitsu, judo, wrestling, and learning. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

A decade ago, a Microsoft developer might have been defined by the fact that they built C# applications on Windows. Today, a Microsoft developer is just as likely to be writing JavaScript for Linux. The company has repositioned itself to focus on cloud services, SaaS products, and enterprise artificial intelligence. Jason Young and Carl Schweitzer host the MS Dev Show, a popular podcast about Microsoft developers and technologies. On their show, they explore the rapidly expanding marketplace of services on Microsoft Azure and talk to experts about how these services are built and what they are used for. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Building a startup requires constant evaluation of tradeoffs. At the earliest stage, the founders evaluate different ideas. Once an idea is settled on, the company develops strategies for finding early customers and growing. As the company develops traction, the operators consider ways to scale further or partner with an acquirer. Joseph Jacks and Greg Koberger are two founders who have been on the show previously. JJ started Kismatic, the earliest company completely focused on Kubernetes. Kismatic was acquired by Apprenda last year. Greg runs Readme.io, a company that provides beautiful documentation as a service. In this episode, Greg and Joe share their thoughts on running and scaling startups–engineering concerns, scaling strategies, and discussions of what to build and what to buy. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Access to education is something everyone strives for but not all achieve–especially education that leads to meaningful and well-paying work. In today’s world where software is eating all sorts of industries, access to a good technical education is still out of the reach of many people. Laboratoria is a social enterprise which teaches women from low-income backgrounds in Peru, Mexico and Chile how to code and helps place them in coding jobs. It was started in Peru by couple Mariana Costa (CEO) and Herman Marin (CTO) along with a friend after they found it difficult to hire developers for a web agency they had started. In today’s episode, Mariana talks to Carl Mungazi about how Laboratoria is using software engineering to change the lives of the women in Latin America whilst also meeting a demand for good technical talent. She discusses the challenges faced by her students, who sometimes spend hours traveling to the school, and her plans for training 10,000 developers over the next five years. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Containers are widely used in projects that have adopted Docker, Kubernetes, or Mesos. Containers allow for better resource isolation and scalability. With all of the adoption of containers, companies like Red Hat, Google, and CoreOS are working on improved standards within the community. Standards are important to this community because of its pace of growth and the number of concurrent projects. If you heard our recent episode about the Linux Kernel’s open source governance, you know that having some rules in place will help encourage the right kind of creativity to thrive. In the world of containers, networking is not well addressed as it is highly environment specific. The Container Networking Interface is an effort to add specifications around how networks of containers can form. Dan Williams is an engineer at Red Hat. In today’s episode, he explores the ideas behind the container networking interface, which gives insights into how the broader community of cloud native technologies is evolving as a whole. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Reinforcement learning is a type of machine learning where a program learns how to take actions in an environment based on how that program has been rewarded for actions it took in the past. When program takes an action, and it receives a reward for that action, it is likely to take that action again in the future because it was positively reinforced. Michal Kempka is a computer scientist work works on VizDoom, an AI research platform for reinforcement learning, with co-creators Marek Wydmuch, Grzegorz Runc, Jakub Toczek, Wojciech Jaśkowski. VizDoom is based on the first-person dungeon game Doom. In VizDoom, an autonomous agent navigates through a maze avoiding enemies. Reinforcement learning is a widely used tool for machine learning, and we will be doing more shows in the future that explain how it works in further detail. Cornell University Library: VizDoom Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

In its most basic definition, machine learning is a tool that makes takes a data set, finds a correlation in that data set, and uses that correlation to improve a system. Any complex system with well-defined behavior and clean data can be improved with machine learning. Several precipitating forces have caused machine learning to become widely used: more data, cheaper storage, and better tooling. Two pieces of tooling that have been open sourced from Google help tremendously: Kubernetes and TensorFlow. Kubernetes is not a tool for machine learning, but it simplifies distributed systems operations, unlocking more time for engineers to focus on things that are not as commodifiable–like tweaking machine learning parameters. TensorFlow is a framework for setting up machine learning systems. Machine learning should affect every aspect of our lives–including tuxedo fitting. Generation Tux is a company that allows customers to rent apparel that historically has required in-person fitting. Using machine learning, they have developed a system that allows customers to get fit for an outfit without entering a brick-and-mortar store. In this episode, Colan Connon and Thomas Bell from Generation Tux join to explain how Generation Tux adopted Kubernetes and TensorFlow, and how the company’s infrastructure and machine learning pipeline work. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software engineers have a skill set that can be applied to solve problems outside of a codebase. Analytical skills can be used to evaluate investment opportunities. Creative thinking can be used to build businesses. Communication skills can be used to build and enhance relationships. John Sonmez is a software engineer who created the Simple Programmer, a community of developers who discuss strategies around software, business, and life. He joined me on the show to discuss these topics and others, as well as his new book The Complete Software Developer’s Career Guide. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] SE Radio: John Sonmez, Marketing Yourself and Managing Your Career Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Every software company backs up critical data sources. Backing up databases is a common procedure, whether a company is in the cloud or on-prem. Backing up virtual machine instances is less common. Rubrik is a company that is known for building backup infrastructure for enterprises. Their main product is an appliance that sits on prem at an enterprise and stores snapshots of virtual machines running within the enterprise. If a virtual machine dies, Rubrik can quickly restore the VM snapshot. The appliance also backs up to the cloud. Kenny To is a founding engineer at Rubrik, and he joins the show to discuss backups and how Rubrik is engineered. Enterprises that start backing up to the cloud through Rubrik start a path towards potentially more cloud services. For enterprises that have not been able to move to the cloud yet, this can be an appealing opportunity. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Shopify is a company that helps customers build custom online storefronts. Shopify has built upon the same Ruby on Rails application since the founding of their business 12 years ago starting with Rails 0.5 and moving all the way to Rails 5. MRuby is a lightweight implementation of the Ruby language. Shopify made the decision to use mruby to allow customers to create custom scripts that are run every time a customer adds items to their cart. However, since mruby was a language implementation that was not widely used, Shopify opted to post a Bug Bounty to the HackerOne bug bounty platform to find security vulnerabilities in their use of mruby. What followed was a payout of over $500,000 as report after report flooded in of security vulnerabilities inside mruby itself. There was so many reports that Shopify made the decision to sandbox the mruby execution into separate processes and decreased the bounty awards by 90%. In this episode, Jeremy Jung interviews Daniel Bovensiepen (BOH-ven-see-pen) about mruby and the Shopify bug bounty. Mruby: http://mruby.org/ The $500,000 release: http://mruby.sh/201703270126.html HackerOne bounty page: https://hackerone.com/shopify-scripts American Fuzzy Lop: http://lcamtuf.coredump.cx/afl/ Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

At Coinbase, security is more important than anything else. Coinbase is a company that allows for storage and exchange of cryptocurrencies. Protecting banking infrastructure is difficult, but in some ways the stakes are higher with Coinbase, because bitcoin is fundamentally unregulated. If a hacker were able to syphon all of the money out of Coinbase accounts, Coinbase would have no recourse–which means this is a more sensitive problem than the regulated banking system, where transactions can often be reversed. Philip Martin is the director of security at Coinbase. He joins the show today to explain why his love of complex and high-stakes security challenges brought him to Coinbase. Philip has some specific points about Coinbase and some more abstract points about security that were very useful to me. This is the third and final episode in our series about Coinbase. Our first two episodes covered the currencies of Coinbase and the fraud prevention techniques the company uses. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Coinbase is a platform for buying and selling digital currency: bitcoin, ethereum, and litecoin. Every payments company deals with fraud, but a cryptocurrency company has a harder job than most payments companies, because bitcoin transactions are anonymous and non-reversible. This is in contrast to a bank, which deals with a regulated, reversible transaction system. Soups Ranjan is the director of data science at Coinbase. In this episode, he walks through the challenges of preventing fraud and describes how machine learning and humans in the loop are used to deal with bad actors. From the data ingestion to the data engineering to the data science, this episode is a great overview of antifraud at Coinbase, and is a nice complement to the presentation that we previously aired from Soups. This is the second episode in our series about Coinbase. Yesterday we discussed how Coinbase makes cryptocurrencies easier to work with. Tomorrow we dive into the security infrastructure of Coinbase. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Cryptocurrencies have seen a surge of value recently. People are starting to see that bitcoin, ethereum, and other currencies are not just for speculation. At worst, they are a store of value–like digital gold. At best, they are a tool for micropayments, smart contracts, and an entire decentralized financial platform. Coinbase is a company for buying and selling cryptocurrencies. This episode is the first of three interviews with different members of Coinbase. In this episode, Linda Xie and Jordan Clifford explain why cryptocurrencies are important, and how products that Coinbase builds make cryptocurrencies easier to use. This is the first in a series of episodes about Coinbase. Tomorrow we will discuss antifraud with Soups Ranjan, director of data science at Coinbase. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software deployment evolves over time. In the 90s, a “deployment” might have meant issuing a new edition of your software via CD-ROM. Today, a deployment is often a multi-stage process. A new software build will undergo automated unit tests and integration tests, before being deployed to users. The deployment might only go out to a small percentage of total users initially, with that percentage going up as the deployment proves not to have bugs. Avi Cavale is the CEO of Shippable, a platform for DevOps. In this episode, we discussed deployments in the context of containers, including a discussion of what has become easier: microservices, feature flagging, and continuous delivery. He also discussed his experience building Shippable. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Apache Kafka is an open-source distributed streaming platform. Kafka was originally developed at LinkedIn, and the creators of the project eventually left LinkedIn and started Confluent, a company that is building a streaming platform based on Kafka. Kafka is very popular, but is not easy to deploy and operationalize. That is why Confluent has built a Kafka-as-a-service product, so that managing Kafka is not the job of an on-call DevOps engineer. Neha Narkhede is the CTO of Confluent and she has been on the show twice before to discuss Kafka. In our last episode, we discussed event sourcing and CQRS with Kafka. In this episode, we explore more common enterprise uses for Kafka, and Neha talks about the engineering complexities of building a managed Kafka-as-a-service product.

A cryptocurrency exchange faces a uniquely difficult fraud problem. A hacker who steals my credentials can initiate a transfer of all my bitcoin to another wallet, and it is a non-reversible, non-identifiable payment. So it is really important to prevent those kinds of fraudulent transactions. At the third Software Engineering Daily Meetup, Coinbase director of data science Soups Ranjan explained how Coinbase stays ahead of fraudsters, and he describes some of the cutting-edge social engineering attacks that are being used to try to steal cryptocurrency–including cell phone takeover attacks. Next week, we will be airing three shows I did on-site at Coinbase–interviews with engineers from three different teams. Check out those shows for a deep dive into cryptocurrency uses, fraud, and infrastructure. Coinbase is an exciting company, and it was a lot of fun getting a panorama for how several parts of the organization function. The next Meetup will be in New York. We don’t know when it will be yet, but sign up to follow us at softwareengineeringdaily.com/meetup. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Airbnb is a company that is driven by design. New user interfaces are dreamed up by designers and implemented for web, iOS, and Android. This implementation process takes a lot of resources, but it used to take even more before the company started using React Native. React Native allows Airbnb to reuse components effectively. React Native works by presenting a consistent model for the user interface regardless of the underlying platform, and emitting a log of changes to that user interface. The underlying platform translates those changes into platform specific code. Leland Richardson is an engineer at Airbnb. In today’s episode, he explains how Airbnb uses React Native, how React Native works, and the future of the platform. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. React Europe Talk Airbnb article on react-sketchapp Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

React Native allows developers to reuse components from one user interface on multiple platforms. React Native was introduced by Facebook to reduce the pain of teams who were rewriting their user interfaces for web, iOS, and Android. Nader Dabit hosts React Native Radio, a podcast about React Native. Nader also trains companies to user React Native through his company React Native Training. In this episode, we explore what a developer can and cannot do with React Native, when a developer needs to use native APIs, and some speculation on the future of React Native. This episode is a good preface for tomorrow’s episode about React Native Interfaces with Leland Richardson of Airbnb. In that episode we will dive deeper into how React Native works and just how big of a change it could be for cross-platform developers. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

“Culture fit” is a term that is used to describe engineers that have the right personality for a given company. In the hiring process, “lack of culture fit” is used to turn away engineers who are good enough at coding but just don’t seem right for the company. As today’s guest Ammon Bartram says, “lack of culture fit” usually means “lack of enthusiasm for what a company does.” Ammon is the co-founder of Triplebyte, a company that is debugging the interviewing process. Triplebyte has interviewed thousands of engineers, and is discovering which aspects of the current hiring process make sense and which are based on superstition, or tradition. We had a great conversation about what culture really means, and how to hire effectively. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

The history of computing can be thought of as a series of ideas rather than objects. From Aristotle’s formalization of the syllogism, to Alan Turing’s model for an all-purpose computing machine, to Satoshi Nakamoto’s distributed transaction ledger–these breakthroughs did not come in the form of polished, tangible objects. In fact, the objects which end up changing computing fundamentally are often built from ideas that seemed trivial at first glance. Chris Dixon is a general partner at venture capital firm Andreessen Horowitz and is the author of the article How Aristotle Created the Computer. One job of a venture capitalist is to be early in identifying the ideas that will evolve into influential, tangible objects. In this article, Chris examined several instances in the history of computing where ideas that looked weird and impractical at first glance ended up being world-changing. Recent examples we discussed are blockchains and neural networks. Chris recently wrote a great article about crypto tokens. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Instacart is a grocery delivery service. Customers log onto the website or mobile app and pick their groceries. Shoppers at the store get those groceries off the shelves. Drivers pick up the groceries and drive them to the customer. This is an infinitely complex set of logistics problems, paired with a rich data set given by the popularity of Instacart. Jeremy Stanley is the VP of data science for Instacart. In this episode, he explains how Instacart’s 4-sided marketplace business is constructed, and how the different data science teams break down problems like finding the fastest route to groceries within a store, finding the best path to delivering groceries from a store to a user, and personalizing recommendations so people can find new items to try. Are you looking for old episodes of Software Engineering Daily, but don’t know how to find the ones that are interesting to you? Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

The code in the Linux kernel changes all the time–11k lines are added, 5.8k lines are removed, and 2k lines are modified DAILY. Linux is an open source operating system that has been worked on for 25 years, and one reason the project is able to move so fast is its governance and release structure. Greg Kroah-Hartman is a fellow at the Linux Foundation, where he takes part in many of the developments in the kernel. This episode was a dive into how open source software gets built at scale and what is in store for the future. The Kubernetes project has drawn comparable attention to the size of Linux, and the Kubernetes project is learning how to manage open source from the Linux community. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Kernel Development Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Modern software applications are often built out of loosely coupled microservices. These services can be written in different languages, by different people, but communication between services needs to be standardized. For this reason, a service proxy is useful. A service proxy is a sidecar container that sits next to a service and facilitates communications with other services. Once every service has a sidecar proxy, that sidecar can be used as a way to communicate with a centralized control plane. The sidecar can report telemetry data to the control plane, and the control plane can be used to set policies across services, such as rules for scaling and load balancing which might vary from service to service. Istio is an open platform to connect, manage, and secure microservices. Istio is a service mesh that uses Envoy service proxies. If all of this sounds confusing–don’t worry, we’ll explain it all in today’s interview with Varun Talwar and Louis Ryan, who work on Istio at Google. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Containers make it easier for engineers to deploy software. Orchestration systems like Kubernetes make it easier to manage and scale the different containers that contain services. The popular container infrastructure powered by Kubernetes is often called “cloud native.” On Software Engineering Daily, we have been exploring cloud native software to get a complete picture of the problems in the space, and the projects that are being worked on as solutions. One area of interest: how should services communicate with each other? What should be standardized? How can you make it easy to identify problems and avoid cascading failures? One solution is the service mesh, a tool that allows services to communicate with each other more safely and effectively. William Morgan was an engineer who helped scale Twitter in the early days when the company was dealing with lots of outages. He was on the show previously to discuss scaling Twitter, and in today’s episode we go into the company that he is running, Buoyant, where he works on building a service mesh called Linkerd. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Scaling Twitter What’s a Service Mesh and Why Do I Need One? Buoyant is hiring: email [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Search engines run our lives. The path we take to information is dictated by Google, Facebook, Amazon, and other forms of search. Search engines feel objective and truthful, but are built through ongoing experimentation and subjective decision making. That’s what has kept Danny Sullivan writing about search engines for twenty years. The content Google prioritizes, the ads that we see, the way that a product review changes how highly a search result appears on a search; these are the topics that Danny studies. He is the founder of Search Engine Land, an invaluable resource for news and updates about search engines and marketing. I’ve been reading Search Engine Land since college, so it was a treat to sit down for a conversation with him. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Professional hackathon programmers travel around the hackathon circuit, winning merchandise and small cash prizes. There are enough hackathons that some programmers actually do this as a full-time job. For example, Peter Ma, a programmer who describes himself as a “rapid prototype specialist.” Peter is a great programmer, and he has received lots of offers to work at big tech companies. What drives him to stay independent and work on hackathon projects? There are other types of corporate hackathons. Many of us are familiar with the hackathon where a manager orders pizza and suggests that everyone stays at the office late fixing bugs. Some hackathons are held for kids, to get them exposed to technologies early on. Lizette Chapman is a reporter at Bloomberg, where she writes about technology, business, and news. I was fascinated by her story about hackathons, and it was great to have her on the show to talk about the characters of the hackathon circuit, and why corporations sponsor hackathons. Lizette has co-hosted the Bloomberg Decrypted podcast in an episode about hackathons. Decrypted is one of my favorite podcasts and I recommend checking it out. These Hackathon Hustlers Make Their Living from Corporate Coding Contests Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software Engineering Daily has been around for almost two years. In this episode Pranay Mohan and Erika Hokanson join me for a reflection on where we have been and where we are going. Pranay was the producer of Software Engineering Daily for the first year, after which he left and went to work at Snapchat. Erika joined the show 9 months ago to work on operations, ad sales, and expansion plans. The thesis of Software Engineering Daily has always been that serious, in-depth material about software provides value. Right now we are a podcast about software engineering. We are planning expansion into a larger media company with video, a mobile app, a desktop platform, more podcasts, and more journalism. Pranay, Erika, and I have crafted the vision for Software Engineering Daily. We want serious, inspiring, technical content to be more widespread. We know that you do too. If you have any suggestions you would like to see from us, you can always email me: [email protected]. It’s already been 500 episodes and I am even more excited about Software Engineering Daily than I was when I began–looking forward to episode 1000. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software architecture address the challenge of communicating and navigating large, complex systems to stakeholders, both technical and non-technical. Over the years software architecture has gone in and out of fashion. Today we discuss why software architecture is important, what it means to have software architecture, and how to properly structure teams and incorporate architecture. Today’s show is guest hosted by David Curry. David sits down with Simon Brown to discuss the importance of having a common language for software systems. Simon is an independent consultant specializing in software architecture, he is the author of Software Architecture for Developers, and founder of Structurizr. If you are interested in hosting a show, check out softwareengineeringdaily.com/host Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

A self-driving car needs to be able to quickly respond to changes in driving conditions. A factory needs to be able to quickly respond to changes in workplace safety. For these kinds of applications, we need processing power closer to the user of the application. If we put all of our application logic in the cloud, we will have to make a network round trip for every request. Servers in the cloud are powerful, but so are the computers at the edge–smartphones, sensors, drones, cars, and on-prem servers. Edge computing is giving us more computation outside the data center. Olivier Bloch works on Microsoft Azure IoT Edge, a set of services for edge computing. Azure IoT Edge includes on-prem versions of Microsoft Azure technologies. Tools that were previously accessible only in the cloud can be deployed and hosted on premise. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

John Looney spent more than 10 years at Google. He started with infrastructure, and was part of the team that migrated Google File System to Colossus, the successor to GFS. Imagine migrating every piece of data on Google from one distributed file system to another. In this episode, John sheds light on the engineering culture that has made Google so successful. He has very entertaining stories about clusterops and site-reliability engineering. Google’s success in engineering is due to extremely high standards, and a culture of intellectual honesty. With the volume of data and throughput that Google responds to, 1-in-a-million events are likely to occur. There isn’t room for sloppy practices. John now works at Intercom, where he is adjusting to the modern world of Google infrastructure for everyone. This conversation made me feel quite grateful to be an engineer in a time where everything is so much cheaper, so much easier, and so much more performant than it was in the days when Google first built everything from scratch. I had a great time talking to John, and hope he comes back on the show again in the future because it felt like we were just scratching the surface of his experience. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

A data-driven organization is more efficient because the company can learn what to focus on. In this episode, Edaena Salinas from The Women in Tech Show interviews Rya Sciban, Product Manager at Periscope Data, who explains the needs of data teams in an organization. We talked about what data analysis is and how this changes as the amount of data grows. Rya explained what analytics clusters are and effective ways of sharing data between the organization. Periscope Data has been successful in retaining women in product and development teams. We talked about effective strategies for this and for having more women in leadership positions. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Deep learning allows engineers to build models that can make decisions based on training data. These models improve over time using stochastic gradient descent. When a model gets big enough, the training must be broken up across multiple machines. Two strategies for doing this are “model parallelism” which divides the model across machines and “data parallelism” which divides the data across multiple copies of the model. Distributed deep learning brings together two advanced software engineering concepts: distributed systems and deep learning. In this episode, Will Constable, the head of distributed deep learning algorithms at Intel Nervana, joins the show to give us a refresher on deep learning and explain how to parallelize training a model. Full disclosure: Intel is a sponsor of Software Engineering Daily, and if you want to find out more about Intel Nervana including other interviews and job postings, go to softwareengineeringdaily.com/intel. Intel Nervana is looking for great engineers at all levels of the stack, and in this episode we’ll dive into some of the problems the Intel Nervana team is solving. Related episodes about machine learning can be found here. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Modern architectures often consist of containers that run services. Those containers scale up and down depending on the demand for the services. These large software systems often use a technique known as event sourcing, where every change to the system is kept in an event log. When an event on the log is processed, several different data stores might be updated in response. In these architectures, containers are interacting with each other. Multiple databases are responding to events in the event log. To connect these systems together, engineers can write small functions to pass data around–you might call these small connecting functions “glue.” Glue functions are a great use for a serverless tool such as AWS Lambda or Google Cloud Functions. As these glue functions grow in popularity, there is an increased need for an open source way to deploy serverless functions. Sebastian Goasgoen works on Kubeless, a serverless execution tool built on top of Kubernetes. In this episode, we explore his take on the “serverless on Kubernetes” problem. This is a great companion episode to yesterday’s interview with Soam Vasani. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected]

Kubernetes is an orchestration system for managing containers. Since it was open sourced by Google, Kubernetes has created a wave of innovation in the infrastructure technology space. Another recent innovation has been the “serverless” execution tools–such as AWS Lambda and Google Cloud Functions. Serverless execution, otherwise known as functions-as-a-service, allows a developer to execute code against cloud servers without specifying which cloud servers they are executing on. Serverless execution is a cheap, flexible resource that any large company wants to have access to. But AWS Lambda and the other popular serverless tools are closed source. This led Soam Vasani to work on Fission, a serverless executor that sits on top of Kubernetes. If you have not heard about either Kubernetes or Serverless, you can check out our previous episodes about either topic. If you are familiar with the two topics, I think you’ll enjoy this episode, in which Soam explains the motivation for serverless on Kubernetes, and the architecture of Fission. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Microsoft’s past is full of stories. It’s early period of corporate domination in the 1990s was followed by a period of government antitrust scrutiny, and a period of unsure product direction. Today, Microsoft’s focus on cloud has allowed the company to regain its footing with a clear trajectory for growth. Since 2002, Richard Campbell has chronicled the Microsoft developer community as co-host of .NET Rocks!, a podcast that was originally about the C# .NET framework. Richard also founded Humanitarian Toolbox, an open source set of tools for assisting disaster relief organizations. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Kubernetes makes it easier for engineering teams to manage their distributed systems architecture. But it’s still not simple to deploy and operate a Kubernetes cluster. Google Container Engine (GKE) is a managed control plane for Kubernetes. Just as developers can use Google App Engine to easily deploy monolithic apps against a platform as a service, we can use Google Container Engine to deploy microservices against a platform as a service. David Aronchick and Chen Goldberg work on Google Container Engine, and they join the show to explain why platform as a service container engines are useful. Google is not the only cloud provider with a container engine–Amazon ECS and Azure Container Engine also allow you to run containers in a managed fashion. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Engineers have plenty to be skeptical about. We look to data sets to give us something resembling objective truth. Some areas of research have so many variables that it is hard to isolate facts. Kyle Polich hosts the popular data science show Data Skeptic, where he examines problems and solutions around data, and he is one of the guests today in our round table discussion. There are some big unanswered questions in our world that might eventually be solved with enough data and the right scientific approach: nutrition, or drug discovery, or image classification. The hiring process is like this. How can you predict whether an engineer will make for a good hire? Ammon Bartram of Triplebyte is working on solving the hiring process for engineering organizations and he is the other guest for this roundtable episode. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.