Software Engineering Daily

Originally published April 17, 2019 Drishti is a company focused on improving manufacturing workflows using computer vision. A manufacturing environment consists of assembly lines. A line is composed of sequential stations along that manufacturing line. At each station on the assembly line, a worker performs an operation on the item that is being manufactured. This type of workflow is used for the manufacturing of cars, laptops, stereo equipment, and many other technology products. With Drishti, the manufacturing process is augmented by adding a camera at each station. Camera footage is used to train a machine learning model for each station on the assembly line. That machine learning model is used to ensure the accuracy and performance of each task that is being conducted on the assembly line. Krish Chaudhury is the CTO at Drishti. From 2005 to 2015 he led image processing and computer vision projects at Google before joining Flipkart, where he worked on image science and deep learning for another four years. Krish had spent more than twenty years working on image and vision related problems when he co-founded Drishti. In today’s episode, we discuss the science and application of computer vision, as well as the future of manufacturing technology and the business strategy of Drishti. Sponsorship inquiries: [email protected]

Originally published April 7, 2017 Engineers in Silicon Valley see a world of constant progress. Our work is creative and intellectually challenging. We are building the future and getting compensated quite well for it. But what if we are actually achieving far less than what is possible? What if, after so many years of high margins, gourmet lunch, and self-flattery, we have lowered our standards for innovation? And if Silicon Valley has been lulled into complacency, what does that say about the rest of the United States? American exceptionalism has faltered and complacency has risen in its wake. Today’s guest Tyler Cowen is an economist and author. His new book The Complacent Class is the final book in a trilogy that describes a decline of American output and a decline in American mindset. Complacent America has lost its ability to assess risk. Children are prevented from playing tag for risk of injury. College students protest against speakers who might present challenging ideas. The number of Americans under 30 who own a business has fallen by 65% since the 1980’s--millennials are too busy going to business school to start businesses. In his books, Tyler weaves together history, philosophy, and contemporary culture. He presents hard data about many different fields, and theorizes about how the trends in those fields relate to each other. He also has a podcast, Conversations with Tyler, and in this episode I tried to mirror his interview style. If you like this episode, you should check out his show--he has interviewed people like Ezra Klein, Peter Thiel, and Kareem Abdul-Jabbar. Sponsorship inquiries: [email protected]

Originally published March 6, 2020 ReactJS developers have lots of options for building their applications, and those options are not easy to work through. State management, concurrency, networking, and testing all have elements of complexity and a wide range of available tools. Take a look at any specific area of JavaScript application development, and you can find highly varied opinions. Kent Dodds is a JavaScript teacher who focuses on React, JavaScript, and testing. In today’s episode, Kent provides best practices for building JavaScript applications, specifically React. He provides a great deal of advice on testing, which is unsurprising considering he owns TestingJavaScript.com. Kent is an excellent speaker who has taught thousands of people about JavaScript, so it was a pleasure to have him on the show. Sponsorship inquiries: [email protected]

Originally published June 21, 2019 Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile technology makes it possible for a smartphone to display rendered 3-D images over a video stream without running out of battery. Ingress was the first game to come out of Niantic, followed by Pokemon Go, but there are other games on the way. Niantic is also working on the Niantic Real World platform, a “planet-scale” AR platform that will allow independent developers to build multiplayer augmented reality experiences that are as dynamic and entertaining as Pokemon Go. Paul Franceus is an engineer at Niantic, and he joins the show to describe his experience building and launching Pokemon Go, as well as abstracting the technology from Pokemon Go and opening up the Niantic Real World platform to developers. Sponsorship inquiries: [email protected]

Originally published July 7, 2017 Airbnb is a company that is driven by design. New user interfaces are dreamed up by designers and implemented for web, iOS, and Android. This implementation process takes a lot of resources, but it used to take even more before the company started using React Native. React Native allows Airbnb to reuse components effectively. React Native works by presenting a consistent model for the user interface regardless of the underlying platform, and emitting a log of changes to that user interface. The underlying platform translates those changes into platform specific code. Leland Richardson is an engineer at Airbnb. In today’s episode, he explains how Airbnb uses React Native, how React Native works, and the future of the platform. Sponsorship inquiries: [email protected]

Originally published October 18, 2019 Apache Kafka was created at LinkedIn. Kafka was open sourced in 2011, when the company was eight years old. By that time, LinkedIn had developed a social network with millions of users. LinkedIn’s engineering team was building a range of externally facing products and internal tools, and many of these tools required a high-throughput system for publishing data and subscribing to topics. Kafka was born out of this need. Over time, Kafka’s importance within LinkedIn has only grown. Kafka plays a central role for services, log management, data engineering, and compliance. LinkedIn might be the biggest user of Apache Kafka in the entire software industry. Kafka has many use cases, and it is likely that they are almost all on display within LinkedIn. Nacho Solis is a senior software engineering manager at LinkedIn, where he helps teams build infrastructure for Kafka, as well as Kafka itself. Nacho joins the show to discuss the history of Kafka at LinkedIn, and the challenges of managing such a large deployment of Kafka. We also talk about streaming, data infrastructure, and more general problems in the world of engineering management. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily. Sponsorship inquiries: [email protected]

Originally published December 9, 2019 Machine learning algorithms have existed for decades. But in the last ten years, several advancements in software and hardware have caused dramatic growth in the viability of applications based on machine learning. Smartphones generate large quantities of data about how humans move through the world. Software-as-a-service companies generate data about how these humans interact with businesses. Cheap cloud infrastructure allows for the storage of these high volumes of data. Machine learning frameworks such as Apache Spark, TensorFlow, and PyTorch allow developers to easily train statistical models. These models are deployed back to the smartphones and the software-as-a-service companies, which improves the ability for humans to move through the world and gain utility from their business transactions. And as the humans interact more with their computers, it generates more data, which is used to create better models, and higher consumer utility. The combination of smartphones, cloud computing, machine learning algorithms, and distributed computing frameworks is often referred to as “artificial intelligence.” Chris Benson is the host of the podcast Practical AI, and he joins the show to talk about the modern applications of artificial intelligence, and the stories he is covering on Practical AI. On his podcast, Chris talks about everything within the umbrella of AI, from high level stories to low level implementation details. Sponsorship inquiries: [email protected]

Originally published September 17, 2019 Ever since Apache Kafka was open sourced from LinkedIn, it has been used to solve a wide variety of problems in distributed systems and data engineering. Kafka is a distributed messaging queue that is used by developers to publish messages and subscribe to topics with a certain message type. Kafka allows information to flow throughout a company such that multiple systems can consume the messages from a single sender. In previous shows, we have covered design patterns within Kafka, Kafka streams, event sourcing with Kafka, and many other subjects relating to the technology. Kafka is broadly useful, and new strategies for using Kafka continue to emerge as the open source project develops new functionality and becomes a platform for data applications. In today’s episode, Tim Berglund returns to Software Engineering Daily for a discussion of how applications are built today using Kafka--including systems that are undergoing a refactoring, data engineering applications, and systems with a large number of communicating services. If you are interested in learning more about how companies are using Kafka, the Kafka Summit in San Francisco is September 30th - October 1st. Companies like LinkedIn, Uber, and Netflix will be talking about how they use Kafka. Full disclosure: Confluent (the company where Tim works) is a sponsor of Software Engineering Daily. Sponsorship inquiries: [email protected]

Originally published January 25, 2019 When TensorFlow came out of Google, the machine learning community converged around it. TensorFlow is a framework for building machine learning models, but the lifecycle of a machine learning model has a scope that is bigger than just creating a model. Machine learning developers also need to have a testing and deployment process for continuous delivery of models. The continuous delivery process for machine learning models is like the continuous delivery process for microservices, but can be more complicated. A developer testing a model on their local machine is working with a smaller data set than what they will have access to when it is deployed. A machine learning engineer needs to be conscious of versioning and auditability. Kubeflow is a machine learning toolkit for Kubernetes based on Google’s internal machine learning pipelines. Google open sourced Kubernetes and TensorFlow, and the projects have users AWS and Microsoft. David Aronchick is the head of open source machine learning strategy at Microsoft, and he joins the show to talk about the problems that Kubeflow solves for developers, and the evolving strategies for cloud providers. David was previously on the show when he worked at Google, and in this episode he provides some useful discussion about how open source software presents a great opportunity for the cloud providers to collaborate with each other in a positive sum relationship.

Originally published December 20, 2018 Ten years ago, there was a distinction between “backend” and “frontend” developers. A backend developer would be managing the business logic and database transactions using Ruby on Rails or Java. A frontend developer would be responsible for implementing designs and arranging buttons using raw HTML and JavaScript. Today, developers can build entire applications in JavaScript. Developers who spent their early career developing frontend JavaScript skills are finding themselves with a surprising amount of power. With NodeJS providing a backend framework and React, Vue, or Angular on the frontend, a single JavaScript developer can write all the code for a whole application—hence the rise of the “full stack developer”. At the same time, the cloud infrastructure is becoming easier to use. Backend-as-a-service simplifies the frustrations of deploying your application, and standing up a database. GraphQL improves the relationship between the frontend and the backend. And futuristic technologies like WebAssembly and web virtual reality are promising to make a JavaScript engineer’s life even more interesting. Adam Conrad is an engineer and a writer for Software Engineering Daily. In recent articles, he has documented the changing nature of the frontend, including JavaScript engines, virtual reality, and how mature corporations are using React and GraphQL. He joins the show to share his perspective on what is changing in the frontend—and how full stack JavaScript engineers can position themselves for future success in a quickly changing market.

Originally published May 16, 2019 React is a set of open source tools for building user interfaces. React was open sourced by Facebook, and includes libraries for creating interfaces on the web (ReactJS) and on mobile devices (React Native). React was released during a time when there was not a dominant frontend JavaScript library. Backbone, Angular, and other JavaScript frameworks were all popular, but there was not any consolidation across the frontend web development community. Before React came out, frontend developers were fractured into different communities for the different JavaScript frameworks. After Facebook open sourced React, web developers began to gravitate towards the framework for its one-way data flow and its unconventional style of putting JavaScript and HTML together in a format called JSX. As React has grown in popularity, the React ecosystem has developed network effects. In many cases, the easiest way to build a web application frontend is to compose together open source React components. After seeing the initial traction, Facebook invested heavily into React, creating entire teams within the company whose goal was to improve React. Dan Abramov works on the React team at Facebook and joins the show to talk about how the React project is managed and his vision for the project.

Originally published August 28, 2019 Kent Beck is a legendary figure in the world of software engineering. Kent was an early advocate of Test-Driven Development (TDD), and popularized the idea of writing unit tests before writing code that would satisfy those unit tests. A unit test isolates and tests a small piece of functionality within a large piece of software. Practitioners of Test-Driven Development write tens or hundreds of tests in order to cover a large variety of cases that could potentially occur within their software. When Kent Beck joined Facebook in 2011, he was 50 years old and thought he had seen everything in the software industry. During Facebook Boot Camp, Kent started to realize that Facebook was very different than any other company he had seen. Facebook Boot Camp is the six-week onboarding process that every new hire learns about the software practices of the company. After graduating Facebook Boot Camp, Kent began to explore Facebook’s codebase and culture. He found himself rethinking many of the tenets of software engineering that he had previously thought were immutable. Kent joins the show to discuss his time at Facebook, and how the company’s approach to building and scaling products thoroughly reshaped his beliefs about software engineering. Sponsorship inquiries: [email protected]

Originally published April 3, 2017 A hedge fund is a collection of investors that make bets on the future. The “hedge” refers to the fact that the investors often try to diversify their strategies so that the direction of their bets are less correlated, and they can be successful in a variety of future scenarios. Engineering-focused hedge funds have used what might be called “machine learning” for a long time to predict what will happen in the future. Numerai is a hedge fund that crowdsources its investment strategies by allowing anyone to train models against Numerai’s data. A model that succeeds in a simulated environment will be adopted by Numerai and used within its real money portfolio. The engineers who create the models are rewarded in proportion to how well the models perform. Xander Dunn is a software engineer at Numerai and in this episode he explains what a hedge fund is, why the traditional strategies are not optimal, and how Numerai creates the right incentive structure to crowdsource market intelligence. This interview was fun and thought provoking--Numerai is one of those companies that makes me very excited about the future.

Originally published March 31, 2017 Brendan Eich created the first version of JavaScript in 10 days. Since then JavaScript has evolved, and Brendan has watched the growth of the web give rise to new and unexpected use cases. Today Brendan Eich is still pushing the web forward across the technology stack with his involvement in the WebAssembly specification and the Brave browser. For all of its progress, JavaScript struggles to run resource-intensive programs like complex video games. With JavaScript falling short on its charge to be the “assembly language for the web” the four major browser vendors started collaborating on the WebAssembly project to allow programming languages a faster, lower level compile target when deploying to the web. Brendan is the CEO of Brave which aims to provide a faster and safer browsing experience by blocking ads and trackers by default in a new browser. The Brave browser is also helping publishers monetize in interesting new ways while also giving a share of ad revenue to its users. Caleb Meredith is the host of this show. He previously guest hosted a popular episode on Inferno, a fast, React-like JavaScript framework. As we bring on more guest hosts, please send us feedback. We want to know what every host is doing well, and what we can improve on.

Originally published September 4, 2018 TIBCO was started in the 90’s with a popular message bus product that was widely used by finance companies, logistics providers, and other systems with high throughput. As TIBCO grew in popularity, the company expanded into other areas through products it developed in-house as well as through acquisitions. One acquisition was Jaspersoft, a business intelligence data platform. When TIBCO acquired Jaspersoft in 2014, the architecture was a monolithic Java application. Around this time, customer use cases were shifting from centralized reporting to real-time, embedded visualizations. The use case of the Jaspersoft software was becoming less centralized and less monolithic and the software architecture needed to change in order to reflect that. Jan Schiffman is a VP of engineering at TIBCO and Sherman Wood is a director at TIBCO. They join the show to discuss the process of migrating a large Java monolith to a composable set of services. Breaking up a monolith is not an easy process--nor is it something that every company should do just because they have a monolith. In some cases, a monolith is just fine. Jan and Sherman explain why the business use case for why the Jaspersoft monolith needed to be refactored, and their approach to the refactoring. We also talk through the modern use cases of embedded analytics and the interaction between business analysts and data engineers. At a higher level, we discuss the lessons they have learned from managing a large, complex refactoring. Full disclosure: TIBCO is a sponsor of Software Engineering Daily.

Osquery is a tool for providing visibility into operating system endpoints. It is a flexible tool developed originally at Facebook. Ganesh Pai is the founder of Uptycs, a company that uses Osquery to find threats and malicious activity occurring across nodes. Ganesh joins the show to talk about Osquery usage and his work on Uptycs. Sponsorship inquiries: [email protected]

Data science is a collaborative field. Collaboration requires sharing the artifacts that data scientists are working on, such as Jupyter Notebooks and SQL tables. Hex is a platform for improving sharing across data science workflows. Caitlin Colgrove and Barry McCardel are founders of Hex and they join the show to discuss what they have built. Sponsorship inquiries: [email protected]

Border Gateway Protocol is a protocol designed for routing and reachability between autonomous systems on the internet. BGPmon is a tool for assessing the routing health of your network, which allows for a network administrator to understand network stability and risk of data. Andree Toonk is the founder of BGPmon and joins the show to talk about BGP, how to monitor routing data, and his work at Cisco. Sponsorship inquiries: [email protected]

Business intelligence is crucial for both internal and external applications at any company. There is a wide array of proprietary BI tools. Today, there is an increasing number of options for open source business intelligence, one of which is CubeJS. CubeJS is an open source analytical API platform for building BI. Artyom and Pavel from CubeJS join the show to talk about what they have built and their vision for the platform. Sponsorship inquiries: [email protected]

For several years, we have had the ability to create artificially generated text articles. More recently, audio and video synthesis have been feasible for artificial intelligence. Rosebud is a company that creates animated virtual characters that can speak. Users can generate real or fictional presenters easily with Rosebud. Dzmitry Pletnikau is an engineer with Rosebud and joins the show to talk about the technology and engineering behind the company. Sponsorship inquiries: [email protected]

An instruction set defines a low level programming language for moving information throughout a computer. In the early 1970’s, the prevalent instruction set language used a large vocabulary of different instructions. One justification for a large instruction set was that it would give a programmer more freedom to express the logic of their programs. Many of these instructions were rarely used. Think of your favorite programming language (or your favorite human language). What percentage of words in the vocabulary do you need to communicate effectively? We sometimes call these language features “syntactic sugar”. They add expressivity to a language, but may not improve functionality or efficiency. These extra language features can have a cost. Dave Patterson and John Hennessy created the RISC architecture: Reduced Instruction Set Compiler architecture. RISC proposed reducing the size of the instruction set so that the important instructions could be optimized for. Programs would become more efficient, easier to analyze, and easier to debug. Dave Patterson’s first paper on RISC was rejected. He continued to research the architecture and advocate for it. Eventually RISC became widely accepted, and Dave won a Turing Award together with John Hennessy. Dave joins the show to talk about his work on RISC and his continued work in computer science research to the present. He is involved in the Berkeley RISELab and works at Google on the Tensor Processing Unit. Machine learning is an ocean of new scientific breakthroughs and applications that will change our lives. It was inspiring to hear Dave talk about the changing nature of computing, from cloud computing to security to hardware design.

Originally published July 27, 2018 React Native allows developers to reuse frontend code between mobile platforms. A user interface component written in React Native can be used in both iOS and Android codebases. Since React Native allows for code reuse, this can save time for developers, in contrast to a model where completely separate teams have to create frontend logic for iOS and Android. React Native was created at Facebook. Facebook itself uses React Native for mobile development, and contributes heavily to the open source React Native repository. In 2016, Airbnb started using React Native in a significant portion of their mobile codebase. Over the next two years, Airbnb saw the advantages and the disadvantages of adopting the cross platform, JavaScript based system. After those two years, the engineering management at Airbnb came to the conclusion to stop using React Native. Gabriel Peal is an engineer at Airbnb who was part of the decision to move off of React Native. Gabriel wrote a blog post giving the backstory for React Native at Airbnb, and he joins the show to give more detail on the decision.

October 1, 2019 The development of self-driving cars is one of the biggest technological changes that is under way. Across the world, thousands of engineers are working on developing self-driving cars. Although it still seems far away, self-driving cars are starting to feel like an inevitability. This is especially true if you spend much time in downtown San Francisco, where you will see a self-driving car being tested every day. Much of the time, that self-driving car will be operated by Cruise. Cruise is a company that is building a self-driving car service. The company has hundreds of engineers working across the stack, from computer vision algorithms to automotive hardware. Cruise’s engineering requires engineers who can work with cloud tools as well as low-latency devices. It also requires product developers and managers to lead these different teams. The field of self-driving is very new. There is not much literature available on how to build a self-driving car. There is even less literature on how to manage a team of engineers that are building, testing, and deploying software and hardware for real cars that are driving around the streets of San Francisco. Mo Elshenawy is VP of engineering at Cruise, and he joins the show to talk about the engineering that is required to develop fully self-driving car technology, as well as how to structure teams to align the roles of product design, software engineering, testing, machine learning, and hardware.

Originally published May 14, 2018 The Kubernetes ecosystem consists of enterprises, vendors, open source projects, and individual engineers. The Cloud Native Computing Foundation was created to balance the interests of all the different groups within the cloud native community. CNCF has similarities to the Linux Foundation and the Apache Foundation. CNCF helps to guide open source projects in the Kubernetes ecosystem--including Prometheus, Fluentd, and Envoy. With the help of the CNCF, these projects can find common ground where possible. KubeCon is a conference organized by the Cloud Native Computing Foundation. I attended the most recent KubeCon in Copenhagen. KubeCon was a remarkably well-run conference--and the attendees were excited and optimistic. As much traction as Kubernetes has, it is still very early days and it was fun to talk to people and forecast what the future might bring. At KubeCon, I sat down with Chris Aniszczyk and Dan Kohn, who are the COO and director of the CNCF. I was curious about how to scale an organization like the CNCF. In some ways, it is like scaling a government. Kubernetes is growing faster than Linux grew, and the applications of Kubernetes are as numerous as those of Linux. Different constituencies want different things out of Kubernetes--and as those constituencies rapidly grow in number, how do you maintain diplomacy among competing interests? It’s not an easy task, and that diplomacy has been established by keeping in mind lessons from previous open source projects.

Originally published January 10, 2020 Slack is a messaging platform for organizations. Since its creation in 2013, Slack has quickly become a core piece of technology used by a wide variety of technology companies, groups, and small teams. The messages that are sent on Slack are generated at a very high volume, and are extremely sensitive. These messages must be stored on Slack’s servers in a way that does not risk a message from one company accidentally being accessible to another company. The messages must be highly available, and they also must be indexed for search. When Slack was scaling, the company started to encounter limitations in its data infrastructure that the company was unsure how to solve. During this time, Josh Wills was the director of data engineering at Slack, and he joins the show to retell the history of his time at Slack, and why the problem of searching messages was so hard. Josh also provides a great deal of industry context around how engineers from Facebook and Google differ from one another. When Slack was starting to become popular, the company quickly began to attract engineers from both of those companies. Facebook and Google have distinct solutions for how they have tackled the problems of data engineering.

GitHub manages a large API surface for both internal and external developers. This API surface has been migrated from purely RESTful requests to GraphQL. GraphQL is a newer request language for data fetching with fewer round trips. Marc-Andre Giroux works at GitHub and is the author of Production Ready GraphQL. He joins the show to talk about GraphQL across the industry, and specifically at GitHub. Sponsorship inquiries: [email protected]

Infrastructure at Spotify runs at high speeds. Developers work autonomously, building and deploying services all the time. Backstage is an open source platform built at Spotify that allows developers to build portals for making sense of their infrastructure. Backstage developer portals are powered by a central service catalog, with centralized services and streamlined development. Stefan Alund joins the show to explain how Backstage works and their role in developing it. Sponsorship inquiries: [email protected]

The JavaScript ecosystem has millions of packages. How do you choose from those packages to find the best in breed for your projects? OpenBase is a system for searching and discovering JavaScript packages. OpenBase includes reviews, insights, and statistics around these JavaScript packages. Lior Grossman is a founder of OpenBase, and joins the show to talk about the JavaScript ecosystem and what he is building. Sponsorship inquiries: [email protected]

Data leaks can cause privacy violations and other cloud security vulnerabilities. Visibility and control of cloud resources can help secure data and ensure compliance and governance. Open Raven is a system for discovering and classifying sensitive data in a public cloud, and assuring compliance and governance. Dave Cole is a founder of Open Raven, and he joins the show to talk through data protection and what he has built with Open Raven. Sponsorship inquiries: [email protected]

Banking and money management are at the core of many modern applications. Payment operations teams work to enable the transfer of funds between different bank accounts, and to track the movement of those funds effectively. Modern Treasury is a company that builds payment operations APIs. Sam Aarons works at Modern Treasury and joins the show to talk through the engineering at Modern Treasury. Sponsorship inquiries: [email protected]

Internal tools are often built with Ruby on Rails or NodeJS. Developers create entire full-fledged applications in order to suit simple needs such as database lookups, dashboarding, and product refunds. This internal tooling creates a drain on engineering resources. Retool is a low-code platform for creating internal tools. These internal tools can be written by bizops, marketing, or roles other than engineers. David Hsu is the founder of Retool and joins the show to talk through what he has built. Sponsorship inquiries: [email protected]

Microservices route requests between each other. As the underlying infrastructure changes, this routing becomes more complex and dynamic. The interaction patterns across this infrastructure requires operators to create rules around traffic management. Tobias Kunze Briseno is the founder of Glasnostic, a system for ensuring resilience of microservice applications. Tobias joins the show to talk about microservice routing and traffic management, and what he has built with Glasnostic. Sponsorship inquiries: [email protected]

Netlify is a cloud provider for JAMStack applications. To make those applications more performant, Netlify has built out capacity for edge computing--specifically “edge handlers”. Edge handlers can be used for a variety of use cases that need lower latency or other edge computing functionality. Matt Biilmann Christensen is the CEO of Netlify and joins the show to talk through the engineering behind edge handlers. Sponsorship inquiries: [email protected]

DevOps practices are shared via community, and community manifests at conferences. Unfortunately, conferences are not possible right now due to COVID-19. The world has turned to virtual conferences. All Day DevOps is a 24 hour conference sharing learnings and software strategies around DevOps, starting November 12th. Derek Weeks and Mark Miller are organizers of the conference and they join the show to talk about modern DevOps. Sponsorship inquiries: [email protected]

Static analysis allows for the discovery of issues in a codebase without compiling. There have been many generations of static analysis tools. A newer static analysis tool is DeepSource, which automates code reviews, identifies bug risks, and generates pull requests to fix them. Jai Pradeesh and Sanket Saurav are founders of DeepSource, and join the show to talk through the creation of static analysis tooling, and their work on DeepSource. Sponsorship inquiries: [email protected]

Data science requires data sets to be cataloged and indexed. The data sets are versioned and might be in CSV files in S3, a database, or another data storage system. Splitgraph allows the user to query this data catalog like it is a Postgres database, routing queries to any data set across your catalog. Artjoms Iškovs and Miles Richardson are the founders of Splitgraph and join the show to talk about data cataloging and what they are building with Splitgraph. Sponsorship inquiries: [email protected]

Sendbird is a company that makes chat, voice, and video APIs for developers. The biggest company in this category is arguably Twilio, but Sendbird works at a higher level of abstraction, with an emphasis on developer experience and visual components. John Kim is the CEO of Sendbird and he joins the show to discuss the engineering and competitive positioning of his company. Sponsorship inquiries: [email protected]

Newer machine learning tooling is often focused on streamlining the workflows and developer experience. One such tool is BentoML. BentoML is a workflow that allows data scientists and developers to ship models more effectively. Chaoyu Yang is the creator of BentoML and he joins the show to talk about why he created Bento and the engineering behind the project. Sponsorship inquiries: [email protected]

Data labeling is a major bottleneck in training and deploying machine learning and especially NLP. But new tools for training models with humans in the loop can drastically reduce how much data is required. Humanloop is a platform for annotating text and training NLP models with much less labelled data. Raza Habib, founder of Humanloop, joins the show to to talk about NLP workflows and his work on Humanloop. Sponsorship inquiries: [email protected]

A customer data platform such as Segment allows developers to build analytics and workflows around customer data such as purchases, clicks, and other interactions. These customer data platforms (CDP) are often tightly coupled to an underlying data warehouse technology. Hightouch is a platform that provides an unbundled CDP--a platform that sits on top of your own data warehouse. The Hightouch team joins the show to talk about what they are building and the CDP ecosystem as a whole. Sponsorship inquiries: [email protected]

Fivetran is a company that builds data integration infrastructure. If your company is performing ELT or ETL jobs to move data from one place to another, Fivetran can help with that movement from source to destination. Once the data is moved into a data warehouse, a tool called DBT (data build tool) can be used to transform the data more effectively. We have done shows previously about Fivetran and DBT. In today’s episode, George Fraser of Fivetran returns to discuss the cross section of these two technologies, and what his company is doing around that integration point. Sponsorship inquiries: [email protected]

Staff engineer is a job title that suggests the engineer has deep expertise, and considerable experience. More and more companies are adopting a “staff engineer track” where an engineer can work to become a staff engineer. What is the role of staff engineer? Is it a management role or an individual contributor? What are the expectations and obligations of staff engineer? Will Larson is an experienced engineer who has worked at Stripe and other prominent tech companies. He joins the show to talk about the role of staff engineering, and the material he has written about it. Sponsorship inquiries: [email protected]

The Salesforce Ecosystem has thousands of developers, designers, product people, and entrepreneurs engaging with each other. Salesforce exposes APIs and SDKs that allow people to build infrastructure on top of the Salesforce platform. In a previous episode, we explored how the ecosystem works as a whole. In today’s show, Chuck Liddell joins the show to talk about how developers themselves engage with Salesforce. Chuck is CEO of Valence, a Salesforce AppExchange ISV that adds native integration middleware to Salesforce. Sponsorship inquiries: [email protected]

Pair programming allows developers to partner on solving problems and learn from each other more effectively. Pair programming has become harder to do as remote work has become more prevalent. GitDuck is a tool to enable more effective pair programming. Dragos Fotescu and Thiago Monteiro are the founders of GitDuck, and they join the show to explain what they have built and their motivation behind it. Sponsorship inquiries: [email protected]

Effective data science requires clean data. As data moves through the data pipeline, there may be errors introduced. Errors can also arise from code changes, database migrations, and other forms of data movement. How can you ensure data quality within a fast moving, dynamic data system? Datafold is a company built around data quality management. It allows users to compare tables and databases, as well as automate data QA. Gleb Mezhanskiy is a founder of Datafold and joins the show to talk about the data quality space and what he is building with Datafold. Sponsorship inquiries: [email protected]

Federated learning is machine learning without a centralized data source. Federated Learning enables mobile phones or edge servers to collaboratively learn a shared prediction model while keeping all the training data on device. Mike Lee Williams is an expert in federated learning, and he joins the show to give an overview of the subject and share his thoughts on its applications. Sponsorship inquiries: [email protected]

For all the advances in software development over the years, one area that has seen minimal improvement is the terminal. Typing commands into a black text interface seems antiquated compared to the dynamic, flashy interfaces available in web browsers and modern desktop applications. Fig is a visual terminal assistant with the goal of changing that. Fig sits next to the developer’s normal terminal and enhances the terminal experience. The founders of Fig, Brendan Falk and Matt Schrage, join the show today to discuss how Fig works and why it is useful to have an enhanced terminal. Sponsorship inquiries: [email protected]

Cloud resources can get out of control if proper management constraints are not put in place. Cloud Custodian enables users to be well managed in the cloud. It is a YAML DSL that allows you to easily define rules to enable a well-managed cloud infrastructure giving security and cost optimization. Kapil Thangavelu works on Cloud Custodian and he joins the show to talk about modern cloud management and what he is building with Cloud Custodian. Sponsorship inquiries: [email protected]

Predicting the spread of COVID-19 is not easy. The best methods we have available require us to extrapolate trends from a large volume of data, and this requires the construction of large-scale models. Because of the expertise needed for developing these models, Silicon Valley engineers were brought in to help develop a maintainable model. Two of these engineers are Josh Wills and Sam Shah, and they join the show to talk about the engineering behind the COVID model, and their work to build it. Sponsorship inquiries: [email protected]

Machine learning models require training data, and training data needs to be labeled. Raw images and text can be labeled using a training data platform like Labelbox. Labelbox is a system of labeling tools that enables a human workforce to create data that is ready to be consumed by machine learning training algorithms. The Labelbox team joins the show today to discuss training data and how to label it. Sponsorship inquiries: [email protected]