Software Engineering Daily

Automation will make healthcare more efficient and less prone to error. Today, machine learning is already being used to diagnose diabetic retinopathy and improve radiology accuracy. Someday, an AI assistant will assist a doctor in working through a complicated differential diagnosis. Our hospitals look roughly the same today as they did ten years ago, because getting new technology into the hands of doctors and nurses is a slow process–just ask anyone who has tried to sell software in the healthcare space. But technological advancement in healthcare is inevitable. Cosima Gretton is a medical doctor and a product manager with KariusDX, a company that is building diagnostic tools for infectious diseases. She writes about the future of healthcare, exploring the ways that workflows will change and how human biases could impact the diagnostic process–even in the presence of sophisticated AI.

Loans give people more financial security. If people know that they can receive a loan, they will be more willing to take intelligent risks. A loan can allow for a short-term investment that pays off enough to justify the interest rate on that loan. For the lender, a loan can be a fantastic return on capital–as long as the lendee does not default. When banks were the rulers of the financial infrastructure, most of them would err on the side of caution when it came to lending. They would adhere strictly to credit scores, and a wanting customer would be out of luck if they did not have a credit score, or if their credit score had gotten lower than acceptable. Newer fintech companies are taking advantage of data sources other than credit scores. They are using machine learning in conjunction with these new data sources to find viable lendees who would be overlooked by traditional institutions. Ofer Mendelevitch is the VP of data science at LendUp. He joins the show to explain why loans are important, how LendUp functions, and the machine learning systems that power an intelligent system of lending.

Sensors are being attached to trains, lightposts, and all kinds of factory equipment. Industrial machinery gives off high volumes of data that can be captured, stored, and processed with machine learning in order to improve workflows and ensure safety. Jayson Delancey works at GE, which is building tools and systems to manage large IoT deployments. The full stack for enterprise IoT involves tools for managing thousands of sensors; databases for storing all the data that is coming off of these devices; authentication and authorization systems for enforcing security. There is a lot to do. In this episode, Jayson surveys some of the technology GE is building with Predix, its industrial IoT platform. He also talked about some of the large scale IoT deployments he has seen.

Most products do not sell themselves. Salespeople bridge the gap between a product creation and a customer who purchases it. People can make a good living on the internet selling niche products–if they can find their customers. The process of taking a large group of potential customers and narrowing it down to only the subset of those customers who will buy your product is known as the sales funnel. The sales funnel consists of multiple stages–the first of which is known as “prospecting.” A salesperson doing prospecting is casting a wide net, sending emails to hundreds or thousands of people, looking for anyone who has some small probability of being interested. Without a tool for prospecting, the process can be very labor intensive. Jean-Baptise Escoyez is the CTO at Prospect.io, a tool for sales prospecting. In this episode, we explored the process of building Prospect.io, from the high level product design to the engineering details of how it is implemented. I use Prospect.io to sell two different products so it was enjoyable to find out how one of my favorite tools works.

Applications built in the cloud are often serving requests from all around the world. A user in Hong Kong could have written to a database entry at the moment just before a user in San Francisco and a user in Germany simultaneously try to read from that database. If the user in San Francisco is allowed to see a different database entry than the user in Germany, that database is not strongly consistent. Strongly consistent databases work such that two users who read the same entry at the same time will receive the same result. Weakly consistent or “eventual consistent” databases are suitable for applications where transaction ordering is not important–photo sharing apps and ecommerce shopping carts, for example. Bank accounts, on the other hand, often need to be strongly consistent. CockroachDB is a scalable, survivable, strongly consistent database. Alex Robinson is an engineer at Cockroach Labs and he joins the show to explain the data model for CockroachDB and how it maintains strong consistency.

Religious extremists use technology to recruit vulnerable individuals to a violent cause. Google is developing ways to combat this extremism through its platforms, namely YouTube. When a user looks for inflammatory religious or supremacist content, YouTube’s “Redirect Method” instead sends those users toward anti-terrorist videos. Google’s fight against extremism compelled writer Lochlan Bloom to write an article called “The Coming Battle: AI, Extremism, and the New War of Ideas.” Lochlan joins the show to discuss the societal implications of giant internet providers controlling our information flow. Lochlan is a science fiction writer, most recently of The Wave, a book that mixes existentialism with quantum physics. Our reality is defined by what we observe, and this theme courses through our conversation–from religion to Twitter to artificial intelligence. The Coming Battle: AI, Extremism, and the New War of Ideas

Content websites are supported by advertising. Most of the advertisements around the internet are dynamic ad slots that change depending on the user who visits the site. Those dynamic ad slots are available to a variety of different bidders. For each ad slot, an auction occurs. The highest bidder gets to serve an ad for that slot. Praneet Sharma is the co-founder of Method Media Intelligence, which he founded with Shailin Dhar, who has been on the show several times to discuss his investigations into the world of ad fraud. I wanted to have his partner Praneet on the show to get his perspective on ad fraud and how to clean up the advertising ecosystem. One advance in dynamic advertising that we discussed is header bidding, and an open source library called PrebidJS. When an ad-supported website gets delivered to your web browser, the HTML begins to load and the JavaScript on the page begins to execute. Some of that JavaScript is calling out to advertising networks looking for the highest bidder. Until the page receives a callback for what to put in the ad slots on the page, the page will not finish loading. Sites that do not manage their ad requests appropriately suffer performance issues. Header bidding is a technique to wrap all of the requests to different advertising exchanges in a single serialized blob of code at the top of the page.

The Internet runs on advertising. Advertising is subject to fraud–but then again, so is every system of online transactions. The amount of money lost in electronic payments fraud and ecommerce scamming is probably much greater than what is lost due to ad fraud. So why do we keep covering advertising fraud on Software Engineering Daily? More of our audience needs to know about ad fraud. Few people realize how much fraud there is in online advertising. In previous episodes of Software Engineering Daily, we have explained, how advertising fraud works, why it is absurd and disgraceful, and why nobody talks about. We also cover ad fraud because I personally find it interesting and sometimes hilarious. Those are the same reasons I invited Shailin Dhar to speak at the third Software Engineering Daily Meetup. Shailin has been on the show twice before and he will be on again in the future. He has made it a full time job to expose ad fraud, and he gives a great presentation on the topic in this episode.

Querying a search index for objects similar to a given object is a common problem. A user who has just read a great news article might want to read articles similar to it. A user who has just taken a picture of a dog might want to search for dog photos similar to it. In both of these cases, the query object is turned into a vector and compared to the vectors representing the objects in the search index. Facebook contains a lot of news articles and a lot of dog pictures. How do you index and query all that information efficiently? Much of that data is unlabeled. How can you use deep learning to classify entities and add more richness to the vectors? Jeff Johnson is a research engineer at Facebook. He joins the show to discuss how similarity search works at scale, including how to represent that data and the tradeoffs of this kind of search engine across speed, memory usage, and accuracy. Notes: Jeff’s blog post about similarity search

Augmented reality is coming at us fast. Every large tech company is rumored to be building an AR product. Microsoft HoloLens is already available to developers. Pokemon Go, the most popular augmented reality product today, was made by a company that was spun out of Google. But Apple seems to be ahead of everyone. Apple’s ARKit is a set of tools for developers to build augmented reality applications. The applications people are building with ARKit are remarkable, and two of those early adopters join the show today for an interview. Jesse Bounds and Siyu Song work at Mapbox, a company that makes mapping, navigation, and location search SDKs. Location is natural companion to augmented reality. If I am walking down the street with a pair of augmented reality glasses on, those glasses can augment the world with information based on my location. Because the fit between AR and mapping is so natural, Mapbox has been rapidly experimenting to build up an expertise in AR. As a result, Jesse and Siyu make for great guests to talk about what engineers can build with ARKit today and what might be possible in the future.

When a user experiences an error in an application, the engineers who are building that application need to find out why that error occurred. The root cause of that error may be on the user’s device, or within a piece of server-side logic, or hidden behind a black box API. To fix a complex error, we need a stack trace of contextual information so that we can correlate events across all layers of an application. James Smith is the CEO of Bugsnag, a company that makes crash reporting and error tracking software. In this episode, he describes how to diagnose errors in modern applications. He also explains how the company functions and how Bugsnag itself is built. The product consumes and stores millions of events which makes for a good discussion of software architecture. Full disclosure: Bugsnag is a sponsor of SE Daily.

GatsbyJS is a framework for building web applications for JavaScript. Gatsby’s original goal was to allow users to create super fast static web sites that could be hosted and served efficiently at a low cost. Most web pages have components from a framework like React or Angular that need to render after the user requests them. This rendering can sometimes require additional requests to external data sources, causing the page to take longer to load. Gatsby uses GraphQL to pull in data at build time and pre-render as much of a site as possible using React’s server side rendering. When a page built with Gatsby is served to a user, as much of the page has been rendered as possible, so that the browser can quickly load everything on the page without additional network requests. Kyle Mathews is the creator of GatsbyJS. He joins the show to describe why he created Gatsby–the high level goals and low level engineering decisions. We also discuss how Kyle intends to take Gatsby beyond just an open source project and turn it into a business.

Building and nurturing a developer community is hard work but it is vital for the growth of a country’s technology ecosystem. When communities coalesce around programming languages, tools or programming methods, what follows is a network of conferences, meet ups and other similar events. Juan Pablo Buriticá, VP of Engineering at Splice, has spent the last decade building developer communities in his home country of Colombia and across Latin America as well as running distributed engineering teams. He has helped to launch a number of conferences, meet-ups and more recently, an online meet-up providing advanced technical information for native Spanish speakers. In today’s episode, Juan returns to Software Engineering Daily to speak with Carl Mungazi about the benefits of having a distributed engineering team and hiring developers from developing nations. He discusses how Colombia came to have the largest Spanish-speaking JavaScript community in the world and the importance of good communication when building software with a team of developers based around the world.

Quality assurance testing is a form of testing that closely mirrors user behavior. Sometimes it is manual, sometimes it is automated. Automated QA tests are scripts that validate correct data representation as the application mechanically runs through high-level workflows–like a login page. Manual QA testers act out use cases of an application to see if there are any bugs that were missed during automated test cases. Manual QA testing is often necessary for complex applications where it is not possible to enumerate all potential workflows within a script. Different companies have radically different workflows for QA testing. There are a variety of ticketing systems, testing frameworks, and team chat applications that play a role in a tester’s daily life. QASymphony is a platform for testing tools that integrates with other popular technologies to centralize a QA testing workflow. Jonathan Alexander is the CTO at QASymphony. He’s also the author of Codermetrics: Analytics for Improving Software Teams. He joins the show to discuss the past and present of QA and his strategies for managing the team that is building QASymphony. Thanks to Kevin Wolf for the intro.

Facebook was rapidly outgrowing its infrastructure in 2009. Classic data center design was not up to the task of the rapid influx of new users and data, photos and streaming video hitting Facebook’s servers. A small team of engineers spent the next two years designing a data center from the ground up to be cheaper, more energy efficient, and more ergonomic for the engineers who worked within. That data center design was open sourced in 2011. Intel, Rackspace, and Goldman Sachs were the first three large organizations to join Facebook in the Open Compute Project, an effort to bring the benefits of open source collaboration to data centers. Steve Helvie works on the Open Compute Project and he joins the show to describe how the project has evolved in the last six years–how it has affected data center design and the implications for the future.

Slack is an application for team communication. Users chat across mobile devices, web browsers, and a desktop application, which means Slack has three places to deploy on rather than two. And the desktop apps on Windows, Mac, and Linux are not identical, so Slack has even more places to deploy. With so many different runtime environments, Slack needs to make technology choices that reduce the chance of errors. TypeScript allows for static typing of JavaScript. The extra compilation step checks the types of variables being passed between different places–so the errors will be discovered at compile time. In an untyped world, those errors might occur at runtime. TypeScript also unlocks the ability to put JavaScript code in an IDE, allowing for more efficient development. Felix Rieseberg is a desktop engineer at Slack, and in today’s episode he explains the unique challenges of building Slack, and why the team moved from JavaScript to TypeScript. TypeScript at Slack – engineering blog

Animations make an application more fun and engaging. For most apps, animation is an afterthought. Developers are concerned with getting the functionality right, and designers have enough work to do simply getting icons, text formatting, and page layout correct. There is also the issue of cross-device compatibility. iOS, Android, and web have different ways of doing animation, with no unifying standard–except gifs, and gifs are not interactive, they simply play from start to finish. Airbnb’s emphasis on design makes it the right company to work on the problem of cross-device, interactive animations. Brandon Withrow and Gabriel Peal are engineers who work on Lottie, a library for animations in iOS, Android, web, and React Native. This episode is about how and why Lottie was built, and how Lottie gets used within Airbnb.

JavaScript is moving so fast. It’s not easy to keep up with all of the frameworks, build tools, and packages. No other language spans frontend to backend, mobile to web to server. Sacha Greif is an independent designer and developer most prominent in his roles as co-author of Discover Meteor and community builder at Sidebar.io, a design newsletter with over 35,000 subscribers, and Hacker News Kansai. He is currently best known in the Javascript community as the maintainer of VulcanJS, and for his annual State of Javascript survey which is now open for 2017. In this episode, Shawn Wang guests hosts a discussion about both projects and Sacha’s thoughts on independent web design and development.

The Internet of Things is the concept that traditionally analog objects, like thermostats and lightbulbs, can be given digital guts and connected to the internet to create more value for users. From Nest thermostats to Phillips Hue lightbulbs, these connected things are starting to enter the mainstream. According to recent estimates by Gartner, over eight billion connected “Things” will be in use in 2017, with that number ballooning to over twenty billion by 2020. Jeremy Foster is a Technical Evangelist at Microsoft and the host of Microsoft Virtual Academy’s “Introduction to Azure IoT” course. In this episode, host Jared Porcenaluk joins Jeremy to discuss developing for the Internet of Things.

Serverless computing reduces the cost of using the cloud. Serverless also makes it easy to scale applications. The downside: building serverless apps requires some mindset shift. Serverless functions are deployed to transient units of computation that are spun up on demand. This is in contrast to the typical model of application delivery–the deployment of an application to a server or a container that stays running until you shut it down. Robin Weston develops large projects with AWS Lambda, and he joined me for a discussion of how to build applications for serverless environments and how to do continuous delivery with serverless functions. One big appeal for continuous delivery fans is that serverless deployments are often smaller–the user is deploying something as small as a function. Full disclosure: ThoughtWorks GoCD is a sponsor of Software Engineering Daily. Serverless Architectures and Continuous Delivery by Robin Weston Robin Weston at Pipeline Conf

After raising $18 million, social networking startup Yubl made a series of costly mistakes. Yubl hired an army of expensive contractors to build out its iOS and Android apps. Drama at the executive level hurt morale for the full-time employees. Most problematic, the company was bleeding cash due to a massive over-investment in cloud services. This was the environment in which Yan Cui joined Yubl. The startup did have traction. There were social media stars who would announce on Twitter that they were about to go on Yubl, and Yubl would be hit by an avalanche of traffic. 50,000 users suddenly logging on to interact with their favorite celebrity was a significant traffic spike. How do you deal with a traffic pattern like that? Serverless computing. AWS Lambda allowed the company to scale up quickly in a cost efficient manner. Yan began refactoring the entire backend infrastructure to be more cost efficient, heavily leveraging AWS Lambda. Unfortunately, Yan’s valiant effort was not enough to save the company. But there are some incredible engineering lessons from this episode–how to build cost-effective, scalable infrastructure. It’s also a case study worth looking at if you work at a startup, whether or not you are an engineer. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Quantum computing is based on the system of quantum mechanics. In quantum computing, we perform operations over qubits instead of bits. A qubit is a vector, which can take on many more values than 0 or 1. The technology used to implement quantum computers is advancing such that it has its own Moore’s Law, but it can also leverage the classical advancements of Moore’s Law. If classical computing advances at the exponential pace of 2^n, quantum computing advances at the pace of 2^2^n. Quantum computing will advance technology in ways that will take us by surprise. If things feel like they are moving fast now, just wait until developers have access to quantum processing units. Machine learning, simulated chemical synthesis, and NP-complete problems are ripe for quantum computers. Vijay Pande is a partner at Andreessen Horowitz and a board member at Rigetti Computing, a quantum computer company. In this episode, we explored what software engineers today need to know about quantum computers and some of the application domains that developers will be working on as quantum computers become available. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Continuous delivery is a model for deploying small, frequent changes to an application. In a continuous delivery workflow, code changes that are pushed to a repository set off a build process that spins up a new version of the application. Testing is performed against that new build before advancing it to production, merging it with the existing codebase. Many continuous delivery products are getting built today because it is a wide open space–much like cloud providers or monitoring tools. There are subjective product and engineering decisions to be made depending on the audience for the product. Heroku Flow is a continuous delivery platform built on top of Heroku, a platform as a service. Andy Appleton is an engineer at Heroku and he joins the show to describe how Heroku Flow was built. Two years of work went into the project from initial conception to launch. Full disclosure: Heroku is a sponsor of Software Engineering Daily. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Patents allow individuals and company to lay creative claim for an invention. A patent can provide protection from having its idea being used without giving credit to its creators. Of course, is that patents can be filed and not turned into products, inhibiting innovation. Patents can also be used offensively in a practice known as patent trolling. Large companies like IBM and Google have so many patents that they have trouble keeping track of them all. And if your company has many different hardware and software products, how can you be sure that your patent collection protects you from a patent troll? Nicole Shanahan is the CEO of ClearAccessIP, a product that indexes patents, looks for vulnerabilities in a corporation’s patent strategy, and finds opportunities in a patent collection for further value. The large text corpus of a patent collection is the perfect place to apply machine learning. We discussed the nature of patents, the intersection between law and software, and the product development process of ClearAccessIP. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Wearables are everywhere. In the medical field they are transforming lives. Haiyan Zhang, Innovation Director at Microsoft Research, created a wearable for a young graphic designer that developed Parkinson’s. Parkinson’s is a condition that inhibits movement, and this wearable allows the Parkinson’s patient to write and draw again. Haiyan explained the research process and the technical aspects of how it works. Edaena Salinas of The Women in Tech Show interviewed Haiyan for this episode. They talked about the Internet of Things, the components of these systems and the technical challenges. Haiyan also explained her path from software engineering to design and the process of commercializing products that come from research. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Self-driving cars are here. Fully autonomous systems like Waymo are being piloted in less complex circumstances. Human-in-the-loop systems like Tesla Autopilot navigate drivers when it is safe to do so, and lets the human take control in ambiguous circumstances. Computers are great at memorization, but not yet great at reasoning. We cannot enumerate to a computer every single circumstance that a car might find itself in. The computer needs to perceive its surroundings, plan how to take action, execute control over the situation, and respond to changing circumstances inside and outside of the car. Lex Fridman has worked on autonomous vehicles with companies like Google and Tesla. He recently taught a class on deep learning for semi-autonomous vehicles at MIT, which is freely available online. There was so much ground to cover in this conversation. Most of the conversation was higher level. How do you even approach the problem? What is the hardware and software architecture of a car? I enjoyed talking to Lex, and if you want to hear more from him check out his podcast Take It Uneasy, which is about jiu jitsu, judo, wrestling, and learning. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

A decade ago, a Microsoft developer might have been defined by the fact that they built C# applications on Windows. Today, a Microsoft developer is just as likely to be writing JavaScript for Linux. The company has repositioned itself to focus on cloud services, SaaS products, and enterprise artificial intelligence. Jason Young and Carl Schweitzer host the MS Dev Show, a popular podcast about Microsoft developers and technologies. On their show, they explore the rapidly expanding marketplace of services on Microsoft Azure and talk to experts about how these services are built and what they are used for. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Building a startup requires constant evaluation of tradeoffs. At the earliest stage, the founders evaluate different ideas. Once an idea is settled on, the company develops strategies for finding early customers and growing. As the company develops traction, the operators consider ways to scale further or partner with an acquirer. Joseph Jacks and Greg Koberger are two founders who have been on the show previously. JJ started Kismatic, the earliest company completely focused on Kubernetes. Kismatic was acquired by Apprenda last year. Greg runs Readme.io, a company that provides beautiful documentation as a service. In this episode, Greg and Joe share their thoughts on running and scaling startups–engineering concerns, scaling strategies, and discussions of what to build and what to buy. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Access to education is something everyone strives for but not all achieve–especially education that leads to meaningful and well-paying work. In today’s world where software is eating all sorts of industries, access to a good technical education is still out of the reach of many people. Laboratoria is a social enterprise which teaches women from low-income backgrounds in Peru, Mexico and Chile how to code and helps place them in coding jobs. It was started in Peru by couple Mariana Costa (CEO) and Herman Marin (CTO) along with a friend after they found it difficult to hire developers for a web agency they had started. In today’s episode, Mariana talks to Carl Mungazi about how Laboratoria is using software engineering to change the lives of the women in Latin America whilst also meeting a demand for good technical talent. She discusses the challenges faced by her students, who sometimes spend hours traveling to the school, and her plans for training 10,000 developers over the next five years. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Containers are widely used in projects that have adopted Docker, Kubernetes, or Mesos. Containers allow for better resource isolation and scalability. With all of the adoption of containers, companies like Red Hat, Google, and CoreOS are working on improved standards within the community. Standards are important to this community because of its pace of growth and the number of concurrent projects. If you heard our recent episode about the Linux Kernel’s open source governance, you know that having some rules in place will help encourage the right kind of creativity to thrive. In the world of containers, networking is not well addressed as it is highly environment specific. The Container Networking Interface is an effort to add specifications around how networks of containers can form. Dan Williams is an engineer at Red Hat. In today’s episode, he explores the ideas behind the container networking interface, which gives insights into how the broader community of cloud native technologies is evolving as a whole. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Reinforcement learning is a type of machine learning where a program learns how to take actions in an environment based on how that program has been rewarded for actions it took in the past. When program takes an action, and it receives a reward for that action, it is likely to take that action again in the future because it was positively reinforced. Michal Kempka is a computer scientist work works on VizDoom, an AI research platform for reinforcement learning, with co-creators Marek Wydmuch, Grzegorz Runc, Jakub Toczek, Wojciech Jaśkowski. VizDoom is based on the first-person dungeon game Doom. In VizDoom, an autonomous agent navigates through a maze avoiding enemies. Reinforcement learning is a widely used tool for machine learning, and we will be doing more shows in the future that explain how it works in further detail. Cornell University Library: VizDoom Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

In its most basic definition, machine learning is a tool that makes takes a data set, finds a correlation in that data set, and uses that correlation to improve a system. Any complex system with well-defined behavior and clean data can be improved with machine learning. Several precipitating forces have caused machine learning to become widely used: more data, cheaper storage, and better tooling. Two pieces of tooling that have been open sourced from Google help tremendously: Kubernetes and TensorFlow. Kubernetes is not a tool for machine learning, but it simplifies distributed systems operations, unlocking more time for engineers to focus on things that are not as commodifiable–like tweaking machine learning parameters. TensorFlow is a framework for setting up machine learning systems. Machine learning should affect every aspect of our lives–including tuxedo fitting. Generation Tux is a company that allows customers to rent apparel that historically has required in-person fitting. Using machine learning, they have developed a system that allows customers to get fit for an outfit without entering a brick-and-mortar store. In this episode, Colan Connon and Thomas Bell from Generation Tux join to explain how Generation Tux adopted Kubernetes and TensorFlow, and how the company’s infrastructure and machine learning pipeline work. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software engineers have a skill set that can be applied to solve problems outside of a codebase. Analytical skills can be used to evaluate investment opportunities. Creative thinking can be used to build businesses. Communication skills can be used to build and enhance relationships. John Sonmez is a software engineer who created the Simple Programmer, a community of developers who discuss strategies around software, business, and life. He joined me on the show to discuss these topics and others, as well as his new book The Complete Software Developer’s Career Guide. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] SE Radio: John Sonmez, Marketing Yourself and Managing Your Career Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Every software company backs up critical data sources. Backing up databases is a common procedure, whether a company is in the cloud or on-prem. Backing up virtual machine instances is less common. Rubrik is a company that is known for building backup infrastructure for enterprises. Their main product is an appliance that sits on prem at an enterprise and stores snapshots of virtual machines running within the enterprise. If a virtual machine dies, Rubrik can quickly restore the VM snapshot. The appliance also backs up to the cloud. Kenny To is a founding engineer at Rubrik, and he joins the show to discuss backups and how Rubrik is engineered. Enterprises that start backing up to the cloud through Rubrik start a path towards potentially more cloud services. For enterprises that have not been able to move to the cloud yet, this can be an appealing opportunity. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Shopify is a company that helps customers build custom online storefronts. Shopify has built upon the same Ruby on Rails application since the founding of their business 12 years ago starting with Rails 0.5 and moving all the way to Rails 5. MRuby is a lightweight implementation of the Ruby language. Shopify made the decision to use mruby to allow customers to create custom scripts that are run every time a customer adds items to their cart. However, since mruby was a language implementation that was not widely used, Shopify opted to post a Bug Bounty to the HackerOne bug bounty platform to find security vulnerabilities in their use of mruby. What followed was a payout of over $500,000 as report after report flooded in of security vulnerabilities inside mruby itself. There was so many reports that Shopify made the decision to sandbox the mruby execution into separate processes and decreased the bounty awards by 90%. In this episode, Jeremy Jung interviews Daniel Bovensiepen (BOH-ven-see-pen) about mruby and the Shopify bug bounty. Mruby: http://mruby.org/ The $500,000 release: http://mruby.sh/201703270126.html HackerOne bounty page: https://hackerone.com/shopify-scripts American Fuzzy Lop: http://lcamtuf.coredump.cx/afl/ Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

At Coinbase, security is more important than anything else. Coinbase is a company that allows for storage and exchange of cryptocurrencies. Protecting banking infrastructure is difficult, but in some ways the stakes are higher with Coinbase, because bitcoin is fundamentally unregulated. If a hacker were able to syphon all of the money out of Coinbase accounts, Coinbase would have no recourse–which means this is a more sensitive problem than the regulated banking system, where transactions can often be reversed. Philip Martin is the director of security at Coinbase. He joins the show today to explain why his love of complex and high-stakes security challenges brought him to Coinbase. Philip has some specific points about Coinbase and some more abstract points about security that were very useful to me. This is the third and final episode in our series about Coinbase. Our first two episodes covered the currencies of Coinbase and the fraud prevention techniques the company uses. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Coinbase is a platform for buying and selling digital currency: bitcoin, ethereum, and litecoin. Every payments company deals with fraud, but a cryptocurrency company has a harder job than most payments companies, because bitcoin transactions are anonymous and non-reversible. This is in contrast to a bank, which deals with a regulated, reversible transaction system. Soups Ranjan is the director of data science at Coinbase. In this episode, he walks through the challenges of preventing fraud and describes how machine learning and humans in the loop are used to deal with bad actors. From the data ingestion to the data engineering to the data science, this episode is a great overview of antifraud at Coinbase, and is a nice complement to the presentation that we previously aired from Soups. This is the second episode in our series about Coinbase. Yesterday we discussed how Coinbase makes cryptocurrencies easier to work with. Tomorrow we dive into the security infrastructure of Coinbase. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Cryptocurrencies have seen a surge of value recently. People are starting to see that bitcoin, ethereum, and other currencies are not just for speculation. At worst, they are a store of value–like digital gold. At best, they are a tool for micropayments, smart contracts, and an entire decentralized financial platform. Coinbase is a company for buying and selling cryptocurrencies. This episode is the first of three interviews with different members of Coinbase. In this episode, Linda Xie and Jordan Clifford explain why cryptocurrencies are important, and how products that Coinbase builds make cryptocurrencies easier to use. This is the first in a series of episodes about Coinbase. Tomorrow we will discuss antifraud with Soups Ranjan, director of data science at Coinbase. We’d love to hear your thoughts on this series, and any other suggestions or feedback you have. Send me an email–[email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Software deployment evolves over time. In the 90s, a “deployment” might have meant issuing a new edition of your software via CD-ROM. Today, a deployment is often a multi-stage process. A new software build will undergo automated unit tests and integration tests, before being deployed to users. The deployment might only go out to a small percentage of total users initially, with that percentage going up as the deployment proves not to have bugs. Avi Cavale is the CEO of Shippable, a platform for DevOps. In this episode, we discussed deployments in the context of containers, including a discussion of what has become easier: microservices, feature flagging, and continuous delivery. He also discussed his experience building Shippable. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Apache Kafka is an open-source distributed streaming platform. Kafka was originally developed at LinkedIn, and the creators of the project eventually left LinkedIn and started Confluent, a company that is building a streaming platform based on Kafka. Kafka is very popular, but is not easy to deploy and operationalize. That is why Confluent has built a Kafka-as-a-service product, so that managing Kafka is not the job of an on-call DevOps engineer. Neha Narkhede is the CTO of Confluent and she has been on the show twice before to discuss Kafka. In our last episode, we discussed event sourcing and CQRS with Kafka. In this episode, we explore more common enterprise uses for Kafka, and Neha talks about the engineering complexities of building a managed Kafka-as-a-service product.

A cryptocurrency exchange faces a uniquely difficult fraud problem. A hacker who steals my credentials can initiate a transfer of all my bitcoin to another wallet, and it is a non-reversible, non-identifiable payment. So it is really important to prevent those kinds of fraudulent transactions. At the third Software Engineering Daily Meetup, Coinbase director of data science Soups Ranjan explained how Coinbase stays ahead of fraudsters, and he describes some of the cutting-edge social engineering attacks that are being used to try to steal cryptocurrency–including cell phone takeover attacks. Next week, we will be airing three shows I did on-site at Coinbase–interviews with engineers from three different teams. Check out those shows for a deep dive into cryptocurrency uses, fraud, and infrastructure. Coinbase is an exciting company, and it was a lot of fun getting a panorama for how several parts of the organization function. The next Meetup will be in New York. We don’t know when it will be yet, but sign up to follow us at softwareengineeringdaily.com/meetup. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Airbnb is a company that is driven by design. New user interfaces are dreamed up by designers and implemented for web, iOS, and Android. This implementation process takes a lot of resources, but it used to take even more before the company started using React Native. React Native allows Airbnb to reuse components effectively. React Native works by presenting a consistent model for the user interface regardless of the underlying platform, and emitting a log of changes to that user interface. The underlying platform translates those changes into platform specific code. Leland Richardson is an engineer at Airbnb. In today’s episode, he explains how Airbnb uses React Native, how React Native works, and the future of the platform. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. React Europe Talk Airbnb article on react-sketchapp Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

React Native allows developers to reuse components from one user interface on multiple platforms. React Native was introduced by Facebook to reduce the pain of teams who were rewriting their user interfaces for web, iOS, and Android. Nader Dabit hosts React Native Radio, a podcast about React Native. Nader also trains companies to user React Native through his company React Native Training. In this episode, we explore what a developer can and cannot do with React Native, when a developer needs to use native APIs, and some speculation on the future of React Native. This episode is a good preface for tomorrow’s episode about React Native Interfaces with Leland Richardson of Airbnb. In that episode we will dive deeper into how React Native works and just how big of a change it could be for cross-platform developers. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

“Culture fit” is a term that is used to describe engineers that have the right personality for a given company. In the hiring process, “lack of culture fit” is used to turn away engineers who are good enough at coding but just don’t seem right for the company. As today’s guest Ammon Bartram says, “lack of culture fit” usually means “lack of enthusiasm for what a company does.” Ammon is the co-founder of Triplebyte, a company that is debugging the interviewing process. Triplebyte has interviewed thousands of engineers, and is discovering which aspects of the current hiring process make sense and which are based on superstition, or tradition. We had a great conversation about what culture really means, and how to hire effectively. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

The history of computing can be thought of as a series of ideas rather than objects. From Aristotle’s formalization of the syllogism, to Alan Turing’s model for an all-purpose computing machine, to Satoshi Nakamoto’s distributed transaction ledger–these breakthroughs did not come in the form of polished, tangible objects. In fact, the objects which end up changing computing fundamentally are often built from ideas that seemed trivial at first glance. Chris Dixon is a general partner at venture capital firm Andreessen Horowitz and is the author of the article How Aristotle Created the Computer. One job of a venture capitalist is to be early in identifying the ideas that will evolve into influential, tangible objects. In this article, Chris examined several instances in the history of computing where ideas that looked weird and impractical at first glance ended up being world-changing. Recent examples we discussed are blockchains and neural networks. Chris recently wrote a great article about crypto tokens. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Instacart is a grocery delivery service. Customers log onto the website or mobile app and pick their groceries. Shoppers at the store get those groceries off the shelves. Drivers pick up the groceries and drive them to the customer. This is an infinitely complex set of logistics problems, paired with a rich data set given by the popularity of Instacart. Jeremy Stanley is the VP of data science for Instacart. In this episode, he explains how Instacart’s 4-sided marketplace business is constructed, and how the different data science teams break down problems like finding the fastest route to groceries within a store, finding the best path to delivering groceries from a store to a user, and personalizing recommendations so people can find new items to try. Are you looking for old episodes of Software Engineering Daily, but don’t know how to find the ones that are interesting to you? Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

The code in the Linux kernel changes all the time–11k lines are added, 5.8k lines are removed, and 2k lines are modified DAILY. Linux is an open source operating system that has been worked on for 25 years, and one reason the project is able to move so fast is its governance and release structure. Greg Kroah-Hartman is a fellow at the Linux Foundation, where he takes part in many of the developments in the kernel. This episode was a dive into how open source software gets built at scale and what is in store for the future. The Kubernetes project has drawn comparable attention to the size of Linux, and the Kubernetes project is learning how to manage open source from the Linux community. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Kernel Development Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Modern software applications are often built out of loosely coupled microservices. These services can be written in different languages, by different people, but communication between services needs to be standardized. For this reason, a service proxy is useful. A service proxy is a sidecar container that sits next to a service and facilitates communications with other services. Once every service has a sidecar proxy, that sidecar can be used as a way to communicate with a centralized control plane. The sidecar can report telemetry data to the control plane, and the control plane can be used to set policies across services, such as rules for scaling and load balancing which might vary from service to service. Istio is an open platform to connect, manage, and secure microservices. Istio is a service mesh that uses Envoy service proxies. If all of this sounds confusing–don’t worry, we’ll explain it all in today’s interview with Varun Talwar and Louis Ryan, who work on Istio at Google. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Containers make it easier for engineers to deploy software. Orchestration systems like Kubernetes make it easier to manage and scale the different containers that contain services. The popular container infrastructure powered by Kubernetes is often called “cloud native.” On Software Engineering Daily, we have been exploring cloud native software to get a complete picture of the problems in the space, and the projects that are being worked on as solutions. One area of interest: how should services communicate with each other? What should be standardized? How can you make it easy to identify problems and avoid cascading failures? One solution is the service mesh, a tool that allows services to communicate with each other more safely and effectively. William Morgan was an engineer who helped scale Twitter in the early days when the company was dealing with lots of outages. He was on the show previously to discuss scaling Twitter, and in today’s episode we go into the company that he is running, Buoyant, where he works on building a service mesh called Linkerd. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Scaling Twitter What’s a Service Mesh and Why Do I Need One? Buoyant is hiring: email [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Search engines run our lives. The path we take to information is dictated by Google, Facebook, Amazon, and other forms of search. Search engines feel objective and truthful, but are built through ongoing experimentation and subjective decision making. That’s what has kept Danny Sullivan writing about search engines for twenty years. The content Google prioritizes, the ads that we see, the way that a product review changes how highly a search result appears on a search; these are the topics that Danny studies. He is the founder of Search Engine Land, an invaluable resource for news and updates about search engines and marketing. I’ve been reading Search Engine Land since college, so it was a treat to sit down for a conversation with him. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: [email protected] Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.