Software Engineering Daily

What is the relationship between your brain and your conscious experiences? This is is the fundamental question of the work of Donald Hoffman, a professor of computer science and cognitive science at UC Irvine. When Hoffman was a child, he wondered whether there was a cognitive dividing line between humans and machines, and that curiosity has driven him to his current work–building a mathematical framework which we can use to model consciousness. Humans would like to believe that evolution selects for traits that allow us to understand the world as it is. From his experimental simulations, Hoffman has shown the opposite–that evolution pushes us towards a mistaken version of reality. For listeners who are interested in theories about whether we live in a simulation, this episode is for you. If you are skeptical of the simulation theory then this episode will also be useful for you, as Hoffman gives one of the most nuanced, comprehensive explanations of reality that I have heard. TED Talk The Case Against Reality

Software developers succeed by combining technical ability, communication skills, and well-reasoned philosophies to craft information systems. Where this podcast focuses on the information systems, Developer On Fire focuses on the engineers who built them. Dave Rael started his podcast as a way to overcome fear and access an entrepreneurial side of himself. In his interviews with prominent engineers, Dave identifies patterns and strategies that have enabled his guests to succeed. He also finds the human side–namely the fallibilities and failures–of legendary developers, which makes it easier to imagine ourselves accomplishing the feats of his guests.

Back in 2008, the range of tools that engineers could use to connect computer systems together were getting quite good. Cloud computing was democratizing access to servers. But the telephony ecosystem was still inaccessible to the average developer. If you needed your program to make a phone call and connect a user to a customer service representative, there was no easy way to do that. Twilio was started to make it easy for developers to connect to telephone systems using simple API calls. This has unlocked many important use cases: from Uber’s communication systems to the widespread adoption of 2-factor authentication. In this episode, Twilio VP of product management Pat Malatack joins the show to explain how the company builds and scales the telephony systems that underpin applications which we use every day. We also talked about how Twilio’s culture shapes how engineering proceeds at the company. Full disclosure: Twilio is a sponsor of Software Engineering Daily.

Engineers today have a variety of career options. You could go work for a large corporation, you could raise money and start a startup, you could freelance and move from job to job with freedom–or you could start a business with the goal of quickly becoming profitable. Courtland Allen was a guest on Software Engineering Daily a few months ago, when he discussed Indie Hackers, a platform he built to share the stories of engineers building business on their own and making money. We only touched the tip of the iceberg in our conversation, so I was excited to invite him to the first Software Engineering Daily Meetup, which occurred earlier this month. Today we are republishing his talk, and I would love to hear your feedback on this format. We will be experimenting more with new hosts and formats throughout 2017, and if you have ideas for the show or you are interested in hosting a show, please send me an email. Also–the first Software Engineering Daily Meetup was fantastic–there were ~200 people showing up so we may have to cap attendance on the next one. Please join the Meetup group if you are interested, and we will let you know when we schedule our next event. Courtland Allen slides in PDF.

Many elderly people live with unhealthy levels of isolation. Social isolation is a problem for anybody, but younger people can use technology to alleviate their isolation with tools like Skype and Facebook. How can we bridge the generational gap and give elderly people access to the same technological tools that younger people find easy to use? Voice interfaces are an important new medium for communicating with computers. Amazon and Google are at the forefront of the voice interface movement, but many other devices are emerging. One of these devices is Elliq, from Intuition Robotics. Elliq consists of a microphone voice interface that sits on a table next to a small tablet computer. An elderly person can talk to the voice interface and have it start a voice call with someone on Skype, or see new Facebook photos or YouTube videos. The discussion of Elliq’s hardware is as interesting as that of its backend software. As Itai Mendelsohn of Intuition Robotics explains, Elliq got to market very quickly as a result of the company’s adoption of high-level managed services, like Firebase and Google’s managed machine learning. ELLIQ – The active aging companion

Whatever engineering problem you have right now, the solution is probably not to write a new programming language. But sometimes it does make sense. JetBrains makes IDEs–the interactive development environments that many people code in, like IntelliJ and Webstorm. And all of these IDEs are written in Java. So the JetBrains team is very familiar with Java and the JVM. Since JetBrains spends so much time working on tooling for developers, they also have an intimate understanding of the problems that developers encounter. Kotlin was developed as an alternative to JVM languages such as Java and Scala. In this episode, Hadi Hariri from JetBrains explains why the company decided to build a new JVM language. We discuss many of its features, such as safety, concision, and its ability to compile to JavaScript. Why Kotlin is My Next Programming Language: Ode to the Language You’ve Never Heard Of, by Mike Hearn YouTube: GOTO 2016 Conference, Kotlin – Ready for Production, Hadi Hariri

Translation is a classic problem in computer science. How do you translate a sentence from one human language into another? This seems like a problem that computers are well-suited to solve. Languages follow well-defined rules, we have lots of sample data to train our machine learning models. And yet, the problem has not been solved–largely because languages don’t always follow rules. We have idioms and subtle contextual clues that make it hard to provide a computer with hard and fast rules for translation. Unbabel is a company whose solution to translation puts a human in the loop to correct the error-prone translations that computers often make. In this episode, Vasco Pedro joins the show to explain Unbabel’s approach to translation, its technology stack, and the business applications for translation.

The goals of microservices are the same as what we have pursued in software engineering for decades: isolation, decoupling, maintainability, scalability. The reason that we use the term microservices is not because we have a completely different idea of what a service is than we used to. We use the term microservices because we are signaling that we need to achieve these architectural goals in a different way than we needed to 10-15 years ago. Markus Eisele is a developer advocate at Lightbend. He joins the show to discuss how enterprises are moving from monolithic architectures to microservices architectures, which has been touched on in previous shows. Lightbend makes a framework called Lagom, that suggests a opinionated strategy for moving towards microservices using message passing, CQRS, and other patterns that we explore. Reactive Microservices Architecture

Building a mobile app requires developers to build a separate version for Android and iOS. The approval process for app stores makes it difficult to deploy quickly and iterate in small batches for developers who are making native apps. These frictions cause fewer developers to write mobile apps than we would have if the smartphone platform was unified. Since the early days of the smartphone, different cross platform solutions have been tried, and many developers got burned after they implemented an app that was supposed to work on iOS and Android, when the performance ended up being poor on both. Today, the cross-platform solutions are finally getting good with frameworks like React Native and Ionic. Ionic allows developers to write apps in AngularJS and deploy them to both Android and iOS. In this episode, Adam Bradley from Ionic gives a brief history of cross-platform apps, and explains what is different today–and why cross platform apps are exploding in popularity, particularly in large enterprises.

Urbit is a completely new way of looking at computing. Every user gets a personal server, which runs your apps, wrangles your connected devices, and defines your secure identity. Your urbit presents your whole digital life as a single web service. Urbit feels foreign and confusing for those of us coming from the traditional web because the normal paradigm is to iterate and paper over the problems of the current platform with new things built on top. Curtis Yarvin, the creator of Urbit, argues that the current model is too fundamentally broken for that to work. As he says: “the Internet as a client-server network has won. The Internet as a peer-to-peer network has failed.” This sounds like yet another quirky, overambitious developer side project–but Urbit has serious legs. The github repo has had 51 committers over its four years of activity. Last year, a public crowdsale of Urbit address space raised more than $200,000. Peter Thiel was an early investor in the project, perhaps partly due to the combination of persistence, technical skill, and unusual opinions of Curtis Yarvin. In this episode, Curtis and Galen Wolfe-Pauly join me for a conversation about Urbit–its strange computing platform and the contrarian philosophies that drive its creators.

Over the past few years, React has become the most popular front end JavaScript framework. As React has matured, the open source community around React has identified areas for improvement. Since React itself is too mature to refactor completely, new projects have been started to take the best aspects of React and start from scratch. Inferno is an extremely fast, React-like JavaScript library for building modern user interfaces. Dominic Gannaway is the creator of Inferno and in this episode he joins Caleb Meredith for an interview about Inferno and other React-like JavaScript libraries.

Product managers are responsible for guiding the design and overall functionality of software. The relationship between product managers and engineers is complimentary: a PM is viewing the product from a perspective that is closer to the customer, so the PM often has the responsibility of navigating high-level tradeoffs in the functionality of a product. Suzie Prince is the head of product at ThoughtWorks and she joins the show to discuss how she navigates tradeoffs as a PM. We also explored communication strategies for how PMs can work productively with engineers, and the overall product development process. Full disclosure: SnapCI is a sponsor of Software Engineering Daily.

Medical imaging is used to understand what is going on inside the human body and prescribe treatment. With new image processing and machine learning techniques, the traditional medical imaging techniques such as CT scans can be enriched to get a more sophisticated diagnosis. HeartFlow uses data from a standard CT scan to model a human heart and understand blockages of blood flow using simulations of fluid dynamics. In today’s episode, Razik Yousfi and Leo Grady from HeartFlow describe the data processing pipeline for the company and what their technology stack looks like.

Data visualization tools are required to translate the findings of data scientists into charts, graphs, and pictures. Understanding how to utilize these tools and display data is necessary for a data scientist to communicate with people in other domains. In this episode, Srini Kadamati hosts a discussion with Jake VanderPlas about the Python ecosystem for data science and the different attempts at creating a data visualization library. Jake VanderPlas is the Director of Research for Physical Sciences at the University of Washington’s eScience institute, where he also received his PhD in Astronomy. In addition to contributing to many Python data science libraries like scikit-learn, scipy, numpy, and matplotlib, he’s written multiple books that have been published by O’Reilly and has given many talks on data science tools and techniques. He’s also the co-creator of the Altair project, which is a declarative data visualization library for Python built on the Vega-Lite visualization grammar.

Column-oriented data storage allows us to access all of the entries in a database column quickly and efficiently. Columnar storage formats are mostly relevant today for performing large analytics jobs. For example, if you are a bank, and you want to get the sum of all of the financial transactions that took place on your system in the last week, you don’t want to iterate through every row in a database of transactions. It is more efficient to just look at the column for the amount of money, and ignore things like timestamp and user id. Julien Le Dem co-created Parquet, a file format for storing columnar data on disk. Jacques Nadeau is a VP of Apache Arrow, a format for in-memory columnar representation. They are both part of Dremio, and they join the show to talk about how columnar data is stored, processed, and shared between systems like Spark, Hadoop, and Python.

Free Code Camp is an online learning platform that takes people from knowing nothing about code to having enough knowledge to build software for a living. We have already done a show with Quincy Larson, the founder of Free Code Camp, in which we discussed his motivation for starting the organization. The economics of running a free interactive platform with thousands of users informs the architectural strategy. Free Code Camp does not even have ads, so it needs to be frugal. Berkeley Martinez is the CTO of Free Code Camp, and in today’s episode he discusses how Free Code Camp works from a technical perspective, including how the site’s sandboxed coding environment is built.

Software engineers are often one of the last lines of defense against potentially dangerous and unethical practices. Every software company encounters situations where the line between right and wrong is not clearly drawn. Back in the year 2000 Bill Sourour was asked to code a project to help market a pharmaceutical product in a misleading way. Reflecting on this experience, Bill recently wrote about it on Medium, suggesting that developers should avoid doing things that compromise our ethics in small ways. The more software defines our real world the more relevant the conversation around ethics becomes. In this episode, Bill and I explore the ethics of modern software development. The Code I’m Still Ashamed Of

A company like Pinterest has millions of transactional emails to send to people. The scalability challenges of sending high volumes of email mean that it makes more sense for most companies to use an email as a service product rather than building their own. Chris McFadden is the VP of engineering and cloud operations at SparkPost and he joins the show to explain the architecture of SparkPost’s email as a service product. SparkPost started as an on-premise email technology for large enterprises, and evolved into a SaaS product. In 2014, the company migrated to the cloud, which has changed its infrastructure as well as its operational model. Full disclosure: SparkPost is a sponsor of Software Engineering Daily.

Vulnerabilities exist in every computer system. As a system gets bigger, the number of vulnerabilities magnifies. The web is the biggest, most complex computer system we have–but fortunately, the steps we can take to secure our web applications are often quite simple. Jared Smith is a cyber security research scientist with Oak Ridge National Laboratory. He joined me on the show to discuss web application security, but I really wanted to know his position on some of the more grandiose topics–Stuxnet, our power grid, Russian hacking, and corporate backdoors. This was a wide ranging discussion and I enjoyed it a lot. For a presentation Jared gave at Nodevember about Web Security, check out this YouTube video.

Meetup is an online service that allows people to gather into groups and meet in person. Since 2002, the company has been growing and its technology stack has been changing. Today, they are in the process of migrating to the cloud, using both Amazon Web Services and Google Compute Platform. Yvette Pasqua is the CTO of Meetup and she joins the show to explain how Meetup’s technology stack works and how the teams are organized. The discussion of multiple clouds is particularly interesting–Yvette describes GCP and AWS as both having distinct, well-defined use cases at Meetup.

When a useful new technology comes out, companies that are in a position to adopt that new technology can gain an edge over competitors. As our industry grows and moves faster, these kinds of changes are coming faster–some recent examples are Docker, ReactJS, and Kubernetes. Evolutionary architecture supports incremental, guided change as a first principle along multiple dimensions. A company with an evolutionary architecture is structured to evolve in response to changes inside the company (such as a decision to change the product) or outside the company (such as the emergence of Docker). Neal Ford is an architect at ThoughtWorks and one the creators of the evolutionary architecture concept.

Every digital system has vulnerabilities. Cars can be hacked, locked computers can be exploited, and credit cards can be spoofed. Security researchers make a career out of finding these types of vulnerabilities. Samy Kamkar’s approach to security research is not just about dissection–it’s also about creativity. For many of the technologies he hacks on, Samy open-sources code that summarily describes the vulnerability he has been working on. For example, in his project PoisonTap, Samy open-sourced code that you can run on a $5 Raspberry Pi, and plug into a locked computer to exploit it. Our conversation covered the art of deconstructing technologies for vulnerabilities and Samy’s goals as a security researcher. We also touched on some of the broader issues of modern security.

Self-contained systems is an architectural approach that separates the functionality of a system into many independent systems. Each self-contained system is an autonomous web application, and is owned by one team. Communication with other self-contained systems or 3rd party systems is asynchronous where possible. As Eberhard Wolff explains in this episode, self-contained systems is not the same thing as microservices, but they are not mutually exclusive. Organizations often adopt a mix of architectural ideas, and it is worth understanding these different models so you can decide which of them to apply to your own projects.

Many people find themselves going down a career path that does not bring them satisfaction. A lawyer finds himself constantly working cases he doesn’t care about. A student in medical school gets completely burned out from hospital bureaucracy. An investment banker no longer finds joy in the accounting statements that she used to enthusiastically study. Startups offer a different career path. Within a startup, an employee can often find creativity, limited bureaucracy, and highly variable reward structures. These ingredients make the world of startups both refreshing and intimidating to someone who feels stuck in a career that no longer gives them joy. Breaking Into Startups is a podcast about people coming from non-traditional backgrounds and making their way into startups. Breaking Into Startups is run by Artur and Timur Meyster and Ruben Harris, who join me for a conversation about how they migrated from more traditional careers to startups–and how many other people are doing the same.

In the 1990s, the barriers to starting a company were significant. Not only did you need an idea, you needed $200,000 for servers and Oracle licenses. With cloud computing, the up-front financial costs of getting a company off the ground have been mostly eliminated–but the idea of starting a company is still perceived as risky. The process of building software has changed dramatically in the last twenty years, but many of the challenges of managing a software company remain timeless. How do you hire properly? How do you keep track of a software stack that is growing in complexity? How do you handle dissenting opinions from employees? Mike Wolfe has been building software companies since the 1990s and joins the show to discuss how to build products and manage engineering teams. It’s a wide-ranging discussion including technological trends, interpersonal skills, and startup financing.

A biologist wants to study the genetic makeup of an organism. A pharmaceutical researcher wants to test the effects of an experimental drug. These types of experiments require a deep knowledge of the scientific domain as well as the lab techniques to produce the data that will eventually yield a result. Transcriptic is a robotic biology laboratory that allows you to make requests for experiments to run remotely. Scientists who use Transcriptic do not have to perform wet lab experimentation. The robotic laboratory process has the potential to do for biology what Amazon Web Services did for computing. Max Hodak is a cofounder of Transcriptic and joins me today to explain how a robotic laboratory works. There are a wide variety of challenges, from hardware integration to software reverse engineering–but we also talk about why it is worth it to overcome these challenges, including a discussion of the “reproducibility crisis” that is undermining faith in our scientific experiments.

Application performance monitoring helps an engineer understand what is going on with an application. An application on a single machine is often monitored by inserting bytecode instructions into the application after it has been interpreted. Distributed cloud applications with functionality broken up across multiple servers often use distributed tracing. Andi Grabner from Dynatrace joins today’s show to explain how monitoring software is built, and how engineers use it to solve problems. Monitoring is core to every business, whether the goal is to understand top-level business processes of to dissect and debug a specific engineering problem. And because monitoring is important at every layer of the stack, there is a plethora of monitoring tools for sale.

Online marketplaces and social networks often have a trust and safety team. The trust and safety team helps protect the platform from scams, fraud, and malicious actors. To detect these bad actors at scale requires building a system that classifies every transaction on the platform as safe or potentially malicious. Since every social platform has to build something like this, Smyte decided to engineer trust and safety as a service. Josh Yudaken joins the show today to discuss how Smyte engineered its platform to provide machine learning models for any organization that wants to take advantage of Smyte for its trust and safety. The tools we discuss include Kubernetes, RocksDB, and Kafka, and Smyte is solving some problems that have not been solved before, so this is a great episode for anyone interested in data engineering or fraud detection–or how to use cloud services and open source tools in unique ways.

Every song you hear on the radio is written with a computer. Computer musicians mostly use synthesizers and samples to compose these songs. A sample is a snippet of recorded sound, sometimes taken from a songs, a movie, or another source. The more samples a musician has access to the better. SampleFocus is a platform where musicians upload and download samples to build songs with. Daniel Trostli is the engineer who is building SampleFocus, and he joins the show to discuss how people use samples to write music and how he built SampleFocus. I met Daniel at the Launch Scale event, a conference about building businesses, and when I was having lunch with Daniel, he told me about working on this personal project full-time, and some of the different business models he was exploring. It’s a great episode for anyone who is thinking about turning a side project into a business.

In the last five years, companies started hiring data engineers. A data engineer creates the systems that manage and access the huge volumes of data that are accumulating on cheap cloud servers. As the saying goes, “it’s more expensive to throw out the data than to store it.” Pete Soderling joins the show to discuss the rise of the data engineer, and how that role interacts with data scientists, software engineers, and machine learning experts. Pete runs Hakka Labs, a site that aggregates material for data science and data engineering. He also organizes DataEngConf, a conference for data engineering.

A huge percentage of online advertisements are never seen by humans. They are viewed by bots–automated scripts that are opening web pages in a browser and pretending to be a human. Advertising scammers set up web pages, embed advertisements on those pages, and then pay for bot traffic to come and view those advertisements. This aspect of the internet is bizarre and alarming. Think about it–how much of the Internet economy depends on online advertising? A lot! How many of those advertisements are consumed by robots? Some estimates say as much as 80%. The more time I spend looking at the online advertising industry, the more perplexed and curious I become. Augustine Fou also has this curiosity. He got a PhD at MIT in material science and engineering–but today he spends his time researching advertising fraud and working as an independent consultant.

For many years, software companies have been breaking up their applications into individual services for the purpose of isolation and maintainability. In the early 2000s, we called this pattern “service-oriented architecture”. Today we call it “microservices”. Why did we change that terminology? Did the services get smaller? Not exactly. Jonas Boner suggests that the movement towards cloud and the increased prevalence of mobile changes how we look at these services–so much that we needed to change the terminology necessary to even talk about them. And once the conversation has shifted to “microservices”, what steps do we need to take to implement them properly? The reactive manifesto is a collection of principles for how to build applications. When the reactive manifesto is applied to the idea of microservices, we get reactive microservices, which Jonas and I discuss in today’s episode.

Some tasks are simple, but cannot be performed by a computer. Audio transcription, image recognition, survey completion–these are simple procedures that almost any human could execute, but the machine learning models have not gotten consistent enough to do them accurately. Scale is an API for human labor, created by Lucy Guo and Alexandr Wang. Similar to Amazon Mechanical Turk, Scale sends small, simple tasks to workers who can complete those tasks. Scale provides an interface that is easy for developers to use, unlike Mechanical Turk, which requires a dashboard. Similar to how Stripe allows developers to build software off of payments systems easily, Scale allows developers to build human-driven, manual tasks into their code–which unlocks a wide range of potential applications, which Lucy and Alexandr discussed with me.

Over the next few years, bots will pervade our lives more and more. These smart, conversational text interfaces provide a new way of engaging with the computer systems that we have been mostly interacting with through web and mobile app interfaces for the last decade. Bots are a necessary complement to the voice interfaces of the future, because we don’t always want to talk to the computer, and natural language processing is not yet good enough to always translate our vocal request accurately. Bots are not toys, they aren’t trivial, and they aren’t going away any time soon. Jon Bruner is the host of the O’Reilly Bots Podcast, a show that has quickly become one of my favorites because it covers both theoretical and applied artificial intelligence, while remaining approachable to listen to. Jon hosts it with Pete Skomoroch, who works on a bot startup full time. Jon and Pete were also organizers of O’Reilly Bot Day, which I enjoyed tremendously and got a lot of knowledge from.

Mesos and Kubernetes are tools for distributed systems management. Kubernetes is built with an emphasis on running services, whereas Mesos is commonly used for a wider variety of workloads, including data infrastructure like Spark and Kafka. Mesos can also be used as a platform to provide resource management for Kubernetes. Dharmesh Kakadia is the author of Apache Mesos Essentials, and has spent time studying both Mesos and Kubernetes. He joins the show to explain the strengths of both platforms, and how they can be used together. In addition to dissecting Mesos and Kubernetes, we had a wide-ranging discussion, from cloud providers to software architecture to business strategy. *** If you are in the Bay Area on January 11th, come to the Software Engineering Daily meetup. There will be awesome speakers, food, and a positive atmosphere. Find more information on softwareengineeringdaily.com or at the Software Engineering Daily meetup page.

Advertising fraud is easy, legal, and extremely profitable. A fraudster can set up a website, scrape content from the internet, and run programmatic advertisements against that website. The fraudster can then purchase bot traffic. Those bots will visit the page, consume advertisements, and return profit to the owner of the page. In a past life, Shailin Dhar worked for a company that set up these types of advertising fraud schemes. He was fascinated by the industry in the same way that plenty of people get fascinated with the fast-moving market dynamics that are enabled by modern software. But over time, the novelty wore off and Shailin realized how big the fraud problem is and how much it was hurting people. In today’s episode Shailin and I discuss how bots and poorly aligned incentives lead to systemic failures and significant financial loss for brands who purchase advertising.

Cloud Foundry is an open-source platform as a service. Companies use Cloud Foundry as a control plane to deploy and manage applications. It provides abstractions for microservices and continuous integration. Cornelia Davis joins the show to discuss Cloud Foundry. I interviewed her at the DevOps Enterprise Summit, where enterprises share their stories of improving their culture and their technology stack. Cornelia explained how and why a large enterprise adopts Cloud Foundry, and why they often hire a company like Pivotal to help them as they migrate their infrastructure and change their processes.

Caching is a fundamental concept of computer science. When data is accessed frequently, we put that data in a place where it can be accessed more quickly–we put the data in a cache. When data is accessed less often, we leave it in a place where the access time is slow or expensive. Netflix has a huge variety of data, and a huge variety of access patterns for how that data gets retrieved from storage. In today’s episode, Scott Mansfield gives an overview of Netflix’s caching architecture, including EVCache, the ephemeral, volatile cache built for Netflix’s cloud architecture. As with other episodes about Netflix architecture, this show is a deeply technical case study.

Software Engineering Daily was started a year and a half ago, based on what I learned from my podcasting experience on Software Engineering Radio. Last week, I interviewed Robert Blumen, the editor of Software Engineering Radio, about how that podcast is produced. In today’s episode, Robert interviews me about this podcast. If you are thinking about starting a podcast about software engineering, this podcast will be useful to you. We discuss the processes behind preparing for a show, how to conduct an interview, why people listen to podcasts, and advertisers. I also give some of the history behind Software Engineering Daily–why I started it and what my goals are.

When you are working on a program, a lot of things are going through your head. In some sense, you become part machine when you are programming. Learnable Programming is a concept that facilitates this, by showing developers what the computer is doing in real time, before compiling. In this episode, Josh Varty, co-founder of Code Connect Inc., talks to Edaena Salinas about incorporating concepts from Learnable Programming into Visual Studio and C# by showing developers what the computer is thinking while they are typing. Code Connect lets a developer immediately see what value a variable is taking, and provide test values to troubleshoot a portion of code to understand how different code paths are going to work. This is instead of having to compile and run the full program. If you have ever thought about making developer tools for an IDE or a programming language, this episode provides some great information.

When a human passes away, we create a tombstone as a memorial. Friends and family visit a grave to remember the times they had with that person while they were still alive. Memorial bots are another way to celebrate the life of someone who has passed away. A memorial bot is created by taking the messages sent by a deceased person and passing it through a machine learning model in order to make a bot that replicates the deceased person. Eugenia Kuyda is the CEO of Luka, a company that builds AI products. When her friend Roman Mazurenko suddenly died, she worked with her team to make a bot that replicates his speech patterns. In our interview, we discussed memorial bots, deep learning, and the product Luka is working on–Replika, a personal AI friend for anyone.

When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like a computer scientist. Brian Christian has a background in computer science and philosophy, and is an author of Algorithms to Live By. He joins the show to explain how the same algorithms and data structures we use for our computer programs can be applied to the real world.

There are multiple paths to constructing a piece of software from start to finish. An individual programmer can build an entire product from scratch in a couple days. A giant corporation can commission a project and delegate responsibilities to hundreds of people. An open source community can use the wisdom of the crowds to efficiently build an operating system. Today’s episode is about another path for building software, and my own experience traveling that path. I designed and contracted an app called Adforprize that is now available on the iOS app store. Designing an app and contracting it out to an engineer to build is a worthwhile exercise for most people in the tech industry–whether you are an engineer or not. This post has two main arguments: As with the previous monologue episodes such as 10 Philosophies for Engineers and You Are Not a Commodity, there are no advertisements on this episode. Please email me any feedback you have–[email protected] Everyone has an app idea, and many of them are pretty good. You have probably had a conversation with a friend where one of you says “hey, wouldn’t it be great if there was a 2-sided marketplace for bacon?” Or “hey, wouldn’t it be great if there was an app where users could take a selfie and instantly see what celebrity they resemble the most?” These things would be cool! But most ideas end with the ideation. Who has the time to implement these things? Most of us are busy at work, where we spend our time and mental energy dealing with the technical complexity of our jobs. If you are a “smart creative”, when you get home from your job, you probably don’t want to spend your leisure hours reading iOS documentation and slowly learning to write the necessary code for your bacon marketplace app. Smart creatives who have a busy full-time job often have a hobby with more immediate gratification, like playing an instrument or blogging. Some of us also write apps on the side, but if you don’t already have all the requisite skills for writing an app, the prospect of reading tutorials and documentation can make the feeling of ever publishing an app seem so distant. There are many people listening to this who are better at sales, marketing, design, or product management than they are at programming–they want to build an app. One common narrative is that if these people want to see something built, they need to learn to program, and that’s not true. There is a more fun, fulfilling way to build an app than learning to code in Swift or React Native–contracting out the programming to someone else. Designing an app is a fun exercise. Building a prototype with tools like Sketch, InVision, and Keynote is free and frictionless. Most people listening to this probably spent some time drawing in KidPix or Microsoft Paint or PhotoShop when they were a kid. Most of us know how to draw at some basic level, because we spend all day seeing and understanding images–drawing is second nature to anyone who can understand shapes. Similarly, we all know how to design apps because we spend all day using them. We understand the purpose of a button or a slider or a swipe. But most of us have no idea how to configure the code around this user experience–the buttons and sliders and TextViews. Along with the design process, the business ideation process is also fun. It’s fun to imagine business models and viral loops and ways to drive user engagement. But without the capability to build the app that embodies your business ideas, it is less fun to think about those ideas. And again–most of us don’t relish the idea of struggling through the compiler errors and Stack Overflow posts that an app implementation requires. We’ve all become product designers. We are using our phones all day–most of us have naturally developed a good intuition for how an app should function. Product design ability is naturally acquired by being steeped in technology all day. If you are an employe...

Eight years ago, MongoDB was an internal project at 10gen, a company that was trying to build a platform-as-a-service out of open-source components. The team at 10gen realized that the platform-as-a-service play would be too complex, and difficult to build. Since MongoDB was the most valuable component of that project, they narrowed their focus to this new document-oriented database. In today’s episode, MongoDB CTO Eliot Horowitz describes the history of MongoDB the open source project, as well as the company–which recently released a managed cloud service. Eliot explained how the company has architected the MongoDB Atlas service on top of AWS, and why developers often want a managed service as their database rather than managing database servers themselves. Full disclosure–MongoDB is a sponsor of Software Engineering Daily.

Four years ago, I started volunteering for a popular podcast about software–Software Engineering Radio. For the next two years, I learned about the process of making a quality podcast about engineering. With its emphasis on preparation, timeless engineering principles, and attention to the listener, Software Engineering Radio continues to be one of the most popular shows about engineering. Software Engineering Daily is my effort to bring the quality of SE Radio on a daily basis. In today’s show, I talk to Robert Blumen, the editor of Software Engineering Radio. Robert is responsible for guiding SE Radio, and without my experience producing shows alongside him, Software Engineering Daily probably wouldn’t exist. If you listen to SE Daily, you should definitely subscribe to SE Radio. I still produce content for SE Radio because the process and the community help me get internal feedback.

C# .NET is the framework that is most often used to write software for the Microsoft Windows operating system. For many years, the C# .NET framework was closed source, but Microsoft’s recent push towards open source has led to the creation of .NET Core, a fork of C# .NET composed of a small subset of features from the original C# .NET stack. This episode takes us through a history of .NET, with two program managers who have worked on .NET for many years–Immo Landwerth and Lee Coward. We also explored the present and future of .NET–discussing .NET Core and .NET usage on operating systems other than Microsoft Windows, which is another recent development.

Biology research at Microsoft is focused on three main areas: molecular programming, synthetic biology, and stem cell biology. At the intersection of biology and computing there are implications for health, medicine, and efficient computing techniques. The field of Biological Computation is in its early days, and there is still lots of work to be done. Colin Gravill works in the computational science group at Microsoft Research, and today he explains how Microsoft is investigating biology with the same curiosity and pragmatism that Microsoft investigates computer science. I’d love to do more shows at the border between software and biology–if you have suggestions, please send me an email. [email protected]

Rust is a systems level language that is built to prevent crashes and eliminate data races. A language like C++ gives you high speed and lots of control, but it is easy to have segfaults, data races, and other problems if you aren’t careful. On this spectrum of control versus safety, we can plot other languages like Java, Go, and Haskell–but none of these languages have the unique feature set of Rust. Rust’s concurrency model is built on the foundation of ownership–a piece of data can only be owned by a single thing at a time. In today’s interview, Alex Crichton explains ownership, and the other abstractions that give Rust its unique model of concurrency.

Microservices are a widely adopted pattern for breaking an application up into pieces that can be well-understood by the individual teams within the company. Microservices also allow these individual pieces to be scaled independently and updated in isolation. Past Software Engineering Daily episodes have covered the microservice architectures of Twitter, Netflix, Google, Uber and other companies. In today’s episode, I sit down with Rafael Schloming, who is building tools for microservices at Datawire.

There is a hierarchy of ways to access and store data in a computer system. The cheapest, slowest way to store and retrieve data is disk. On the faster end, we have memory. As we architect systems with increasing complexity, we have additional considerations–network latency, transient compute nodes, and numerous caching layers. Apache Geode is a distributed, in-memory system for the JVM. It provides fast data storage and retrieval. Swapnil Bawaskar is an engineer who works on Geode, and he joins the show today to explain the architecture of Geode, and how Geode is different from other in-memory systems that get used for caching, such as memcache or Redis.