Software Engineering Daily

Frontend web development was simpler in the past–CSS, HTML, and JavaScript were all you needed to know. Today, we have mobile web, React, Angular, PHP, JQuery, and so much more. Marc Grabanski focuses on what he believes is timeless–pure JavaScript. Marc is the founder of Frontend Masters, a training site for developers who want to build quality web interfaces. He joins us today to discuss the shifting nature of front end development.

Nadia Eghbal joins us as the host of today’s show to discuss the past, present, and future of open-source software. This monologue was adapted from Nadia’s article, We’re in a brave, new post open source world. In this episode, Nadia describes the origins of open-source in the Free software movement, its rise to popularity, and today’s golden era where mainstream and popular technologies like React, Spark and Docker are all open source projects. Yet there are several serious issues that threaten the foundation upon which the current open-source renaissance is happening. Nadia discusses what these are, and how to resolve these issues so that we can continue to have a vibrant ecosystem for developers. Thank you to Nadia Eghbal for coming on the show and for standing up for open source software.

When you spend money online, you expect a receipt to come in your email. When you register for a new web site, you need to verify your sign up in your email. These types of emails are called “transactional email” and sending these types of email at scale is a complex engineering task. J.R. Jasperson is the chief architect at SendGrid, a transactional email platform. On this episode, we discuss how email works–from the basics to the massive scale that SendGrid operates on. We also talk about email spam and fraud in detail.

Git is the most popular version control system. If you have been programming for less than a decade, it’s likely that you haven’t used any other method of version control. The git workflow of a software team defines how that team collaborates, builds, and ships software. Tim Pettersen is a developer advocate at Atlassian, where he has been building software around git for many years. He joins us on today’s episode to talk about strategies for git–including branching, merging, continuous integration, and software as a service.

Application delivery has become more complex as software architectures have moved into the cloud. Data center infrastructure has turned into code to be manipulated, and software engineering teams are adjusting their strategies. HashiCorp is a company that builds open-source software for application development and deployment. Mitchell Hashimoto is the founder of HashiCorp, and he joins us to discuss a modern approach to application delivery, and the tools HashiCorp is developing.

Code analysis tools can help a developer understand code. One tool for code analysis is bitHound, which provides code and dependency analysis for NodeJS applications. On today’s episode, we discuss how to use a code analysis tool–and we also talk about how to build one, by breaking down the distributed architecture of bitHound’s backend.

Uber is a transportation company with a high volume of temporal spacial data, constantly being collected from the devices of its users. At any given time, the engineers and data scientists at Uber need to be able to query the system, and understand what is going on with drivers and riders. The unique real-time engineering requirements of Uber lead to an interesting architecture. Danny Yuan joins us today to discuss Uber’s data engineering stack and how the company makes use of its streaming data.

The last editorial we did was 10 Philosophies for Engineers. Listeners enjoyed that episode, so we decided to do another. 10 Philosophies was a collection of beliefs I have about the software engineering industry, and how we can find fulfillment in our work as engineers. The philosophies that I discussed were rooted in my experiences working as a computer science student and as an engineer at several different companies. Before writing any lines of code, I played poker competitively for five years, from the age of 15-20. This was a formative experience. Playing poker changed my perspective on money, risk, and statistics. It has been 7 years since I stopped playing poker full time, but the lessons of the game are still with me. In this editorial, I will discuss some beliefs I have that relate to both poker and software engineering. As with the 10 Philosophies episode, these beliefs my own opinions. If you disagree with them, I would love to know why. At Software Engineering Daily we actually want to know any thoughts you have on our content. We make content for our listeners and our readers. Please tell us how we can improve by emailing us, or by filling out the listener survey. As a poker player becomes a software engineer, certain trends about human-computer interaction become apparent: This post explores each of these trends, explaining why these trends are important to poker players, software engineers, and everyone else. Automated Games In 2008, poker was the perfect sport for human-computer symbiosis. What Tyler Cowen said about freestyle chess also applied to poker: In poker, a human with a statistical “heads-up” display can make decisions that are more mathematically justified than a human without such a tool. Heads-up displays create the poker version of “human-plus-machine teams”. One thesis of Average is Over is that a human will only be employable in the future by finding a career where human reasoning provides defensible value to the problem solving process of a computer. If the human’s responsibilities are not defensible, the human will beobviated. In a subsequent blog post, Cowen addresses the “flip” that can occur when a computational problem no longer requires human assistance: Poker players have been increasingly defeated by machines for the past 10 years. It is no surprise that Google’s AlphaGo has defeated human champion Lee Sedol. If Google decided to beat humans at poker, it would be a trivial exercise for the researchers. Poker seems different than Go or Chess, since there is nondeterminism. You start with two cards, but you don’t know how the board will develop. It would seem that fate is in control, unlike Go and Chess, which have no random elements. With just 4 suits and 13 ranks, a poker game has a trivial branching factor. The nondeterminism is so minimal for a computer to plan around that it is effectively deterministic. Imagine if AlphaGo had to learn to play a version of Go with the following rule: at the beginning of each turn, flip a coin. If you lose the flip, you don’t get to move. Adjusting to this rule would be trivial. That is the magnitude of nondeterminism within poker. Poker, Chess, and Go have small decision spaces. The rules never change, the game pieces never change, there is minimal nondeterminism. A computer can assess a hand of poker as it would a hidden Markov model, but it will take work on par with the AlphaGo team for a computer to be trained to build a model accurately. The job of a professional poker player has been a bad long-term choice for a human to pursue for more than a decade, because it is vulnerable to automation. Games like Go, Poker, and Chess can be automated with machine learning techniques we understand today. The rules, game piece schema, and objectives are easy to define, so these games are ripe for supervised learningand reinforcement learning. Yann LeCun protested against the hype around the AlphaGo victory: Poker is vulnerable t...

Applications can and will crash — it is increasingly important for developers to have visibility into the reasons how and why the crash occurred. James Smith is the guest on the show today, and joins Jeff to discuss why modern applications crash, and how developer products are improving to tighten the gap between QA/support and dev. James is the founder of Bugsnag, an automated crash monitoring system.

Ten years ago, building a highly scalable image delivery service would require millions of dollars in upfront costs, and hours of work configuring hardware server infrastructure. Today, it is possible to bootstrap this type of service, with minimal investment. Today’s episode is about building a content delivery network for images and video. Today’s guest is Itai Lahan, CEO of Cloudinary. We discuss Cloudinary’s early product infrastructure, and how they have evolved as a company since then. We also talk in detail about the venture capital landscape of Silicon Valley today, and how to strategize about raising money. Full disclosure, Cloudinary is a sponsor of Software Engineering Daily.

Memory is king. The cost of memory and disk capacity are both decreasing every year–but only the throughput of memory is increasing exponentially. This trend is driving opportunity in the space of big data processing. Alluxio is an open source, memory-centric, distributed, and reliable storage system enabling data sharing across clusters at memory speed. Alluxio was formerly known as Tachyon. Haoyuan Li is the creator of Alluxio. Haoyuan was a member of the Berkeley AMPLab, which is the same research facility from which Apache Mesos and Apache Spark were born. In this episode, we discuss Alluxio, Spark, Hadoop, and the evolution of the data center software architecture.

Cloud service providers like Amazon, Google, and Microsoft provide both infrastructure as a service and platform as a service. Infrastructure as a service gives developers access to virtual machines, servers, and network infrastructure. Platform as a service is the software that runs on top of that infrastructure as a service–this includes things like Amazon DynamoDB, Microsoft Azure Machine Learning, and Google App Engine. OpenStack is an open-source cloud operating system. Today’s guest is John Purrier, a founder of OpenStack and the CTO of Automic Software.

In today’s episode, Ben Halpern interviews Juan Buritica, VP of Engineering at Ride. They discuss Ride’s migration from a monolith to microservices, the challenges of running a distributed team and preventing developer burnout.

GreyOrange Robotics builds robots for warehouse automation of logistics and ecommerce companies for quicker deliveries. Today’s episode features Akash Gupta, the CTO of GreyOrange.

DevOps has been the subject of many episodes of Software Engineering Daily. And yet–the question we continue to ask ourselves is “what is DevOps?” In order to understand the present, we must look into the past. On today’s show, we break down the history of dev ops with Demir Selmanovic, the lead technical editor at TopTal. From waterfall software development, to the agile manifesto. From lean manufacturing to the productive pinnacle of DevOps nirvana. Demir also wrote a blog post called What the Hell is DevOps, which he posted on the TopTal blog. Full disclosure: TopTal is a sponsor of our show, but this episode doesn’t discuss TopTal very much so hopefully that doesn’t impact the trust of our listeners.

Today’s guest is Calvin French-Owen, the CTO of Segment, a tool that companies use to aggregate their analytics into once place. As Segment has scaled, the company has had to restructure its etire technical architecture. Microservices, containers, Amazon Web Services, and dev ops are a few of the topics that Calvin and I explore in our conversation, so this is a great episode for anyone who is trying to understand the relationships between those different subjects. Segment’s product unifies analytics from different services and puts them into one centralized place. Full disclosure: Segment is a sponsor of Software Engineering Daily. For most of this episode, we don’t even talk about the product, we talk about the back end engineering behind the product.

CodeNewbie is a community of programmers and people learning to code. There are so many people learning about software today, and CodeNewbie gives them a place to hang out, socialize, and become comfortable with the world of software. CodeNewbie has an excellent podcast, and if you like Software Engineering Daily you should check it out. Saron Yitbarek is the guest today, she is the creator of CodeNewbie. In this episode of Software Engineering Daily, Saron sits down to talk about learning to code after she had already had several careers, spanning biology, public radio, and marketing.

Mic.com is a media company focused on news for millennials. Anthony Sessa is the VP of product at Mic.com, and he joins us to talk about the engineering of a modern news organization tailored to young people. We discuss data engineering, frontend technologies, and how to migrate away from WordPress. We also explore how to build a successful media company today, and how millennials want to consume news and software.

Software developers have been socializing on chat rooms for decades. In the nineties, we began using IRC and AOL instant messenger. In the early 2000s, we turned to Google Hangouts and Yammer. Today, we are using modern apps like Slack and Hipchat. On today’s show, we take a deep dive into Gitter, the chat client specifically designed for developers. Our guests are Mike Bartlett and Andrew Newdigate, the creators of Gitter. Gitter is a highly scalable, real time social application for developers to talk about writing their software. This is a great episode that spans topics like front end development, back end distributed systems, how to compete with Slack, and how to scale a chat room to tens of thousands of active users.

Military veterans have the right set of skills to become programmers. Technical expertise, emotional resilience, psychological persistence, and teamwork–these are the qualities of the US army and they are the qualities of the best programmers. More and more veterans who leave the army are becoming coders, and the mission of Operation Code is to help veterans make that transition. Dave Molina is the founder of Operation Code. His organization helps army veterans become coders. Dave came on the show to discuss coding boot camps, military, software education, and what it’s like to become a civilian technologist after spending years in the army.

Building a software business today requires lots of decision making. Building software isn’t just about choosing a programming language, or a framework, or a database. Developers have to choose the right cloud service provider, the best issue tracking service, the best hosted code repository, the clearest data visualization tools. We need a platform to compare the options between different products, all in one place. In today’s episode, we talk to Ayan Barua from Siftery, a platform for software products and the companies who use them. Siftery’s mission is to help technologists find the best products for their job, whether they are a sysadmin or an advertising account manager. My discussion with Ayan covers the question of build versus buy, how to build a recommender system, and how software will be built in the future; this is also a great episode for anyone who is thinking of how to build a two-sided marketplace business.

Software engineering is a deterministic field. We write lines of code, and feed data into that code, expecting to get a certain answer. Computing is deterministic because humans developed it–we understand computers from top to bottom. The same cannot be said about biology. Matt Might is an associate professor with a PhD in computer science. When his son was diagnosed with an extremely rare illness, he was confronted with the uncertainties of human biology. In this episode, we discuss Matt’s quest to solve the puzzle of his son’s disease–computer science meets genetics, on this episode of Software Engineering Daily.

In many ways the engineering job market is broken. Engineers cannot find the right jobs where they are valued properly and enjoy their work. Employers have difficulty filling their positions and finding the right people that fit their cultures. Traditional matchmakers like Monster and generic recruiting agencies don’t cut it — you know this by the generic LinkedIn messages you get looking for a Java expert… when you have JavaScript as a skill on your profile. Hired.com is trying to fix this broken process by using a high-touch approach to recruiting. Hiten Parmar, an engineer and product strategist at Hired joins us today to discuss the issues at hand, as well as how Hired is approaching the solution. We chat about the challenges facing engineers, which skills are in demand, and the inner workings of Hired. Full disclosure: Hired is a sponsor of the show, but we would still have done this interview because we genuinely want to discuss the job market, and figure out how companies are addressing this issue.

Stack Overflow is used by developers to find out how to build software. Stack Overflow is both a tool and a community, and today’s guest Jeff Atwood has made a career out of building tools and communities. As the co-founder of Stack Exchange and Discourse.org, Jeff has been solving the problem of civilized online communication for seven years. In today’s episode of Software Engineering Daily, Jeff Atwood talks about building online communities from the perspective of an engineer as well as a sociologist.

On Software Engineering Daily, we often discuss big data in terms of data engineering and data science. Data engineering is the infrastructure and pipelines that handle massive amounts of data and puts that data in a data lake. With a data infrastructure in place, a data scientist can study the data and take action on it. A data scientist can also create visualizations–which is the subject of today’s episode. Aurelia Moser has a specialty in mapping and data visualization. We discussed how to create effective visualizations of our data, and the tools that can be used to collect and present our data. Whether you work at the New York Times or a small tech company, data visualizations are important, because visualizations can be used to communicate important trends across an organization. Aurelia will also be speaking at the upcoming Strata + Hadoop World Conference in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Big data is yesterday–fast data is now. FiloDB is a reactive columnar OLAP database that is built on Cassandra and Spark. Today’s guest is Evan Chan, creator of FiloDB. In our discussion today, we talk about the use cases of an OLAP data store. Evan explains how to tackle the problem of video analytics–if you have ever found yourself asking how a company like YouTube or Netflix or Ooyala performs analytics on millions of users watching millions of videos, this episode is for you. By combining the database features of Cassandra with the data processing power of Spark, Evan created FiloDB to help solve this type of analytics problem. Evan will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Apache Cassandra can serve as both the real-time data store for online transactional applications, as well as the read-intensive database for data warehousing operations. In order to combine these two use cases into a single database, Apache Cassandra required lots of innovation. In today’s episode of Software Engineering Daily, we discuss the internals of Cassandra. Tim Berglund takes us through how Cassandra performs reads and writes, how Cassandra offers tunable consistency, and what inspirations Cassandra took from the Amazon Dynamo paper as well as the Google BigTable paper. Tim will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Hadoop was created in 2003. In the early years, Hadoop provided large scale data processing with MapReduce, and distributed fault-tolerant storage with the Hadoop Distributed File System. Over the last decade, Hadoop has evolved rapidly, with the support of a big open-source community. Today’s guest is Mike Cafarella, co-creator of Hadoop. Mike takes us on a journey from past to present. Hadoop was based on the Google File System and MapReduce papers, and so Mike and I talk about what it was like to work on a distributed file system in 2004, and the challenges of implementing real software systems based on white papers. We also discuss YARN, and the wave of innovation that YARN enabled within the Hadoop ecosystem. Mike will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

When a company gets big enough, there is so much data to be processed that an entire data engineering team becomes responsible for managing this data and making it available to other teams. Airbnb is one such company. Max Beauchemin works on the data engineering team at Airbnb, where he creates infrastructure and tooling for managing data. In this episode of Software Engineering Daily, we talk about AIrflow, a workflow scheduler that assists in job processing. If you don’t know what a workflow is, or a job, we will explain that in this episode. Max and I also talk about Panoramix, a data slicing and visualization tool that helps data scientists and business analysts understand large volumes of data. Max will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Diagnosing illness today requires the trained eye of a doctor. With machine learning, we might someday be able to diagnose illness using only a data set. Today on Software Engineering Daily, we are joined by David Kale, a researcher at the intersection of machine learning and clinical data. We discuss the machine learning and research techniques he is using to diagnose illnesses using neural networks, and we also talk about the challenges of performing data science in hospitals, where the data is mostly confidential. David will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Ruby on Rails has had the most commits made by its creator, David Heinemeier Hansson. The next most frequent contributor to Rails is Aaron Patterson, our guest on today’s episode of Software Engineering Daily. Topics discussed in today’s episode include Ruby relative to JavaScript, Rails relative to Linux, and DHH relative to Tenderlove.

Microsoft and Amazon are the tech giants of the Pacific Northwest. These two companies shape Seattle technology, and Todd Bishop has been writing about them for over a decade. Todd is the co-founder of GeekWire, an online media company focused on technology in the Pacific Northwest. We talk about the past and present of Microsoft and Amazon, what it means for an engineering company to be good or evil, and what it’s like to be a software journalist.

Socket.io enables realtime bidirectional communication. But what does “realtime” actually mean? Today’s guest is Guillermo Rauch, the creator of Socket.IO, a widely used technology for client server communication. We discuss the nature of real-time apps like Uber and Google Docs, and talk about the API and usage of Socket.IO.

Technology careers are not as accessible to certain demographics, such as women. Sarah Allen is working to change that through education and outreach. Sarah Allen created RailsBridge and Bridge Foundry, a pair of organizations working to make technology more accessible to people who are underrepresented in tech. RailsBridge workshops are a fun and free way to get started with Ruby, Rails or other technologies. RailsBridge was so successful that Sarah created Bridge Foundry, which has an even wider variety of free workshops, from Clojure to Go to Mobile. This is an inspirational episode of Software Engineering Daily, and is a great learning opportunity for anyone interested in building a community.

Continuous integration and deployment are important tools for modern software development. With continuous integration and deployment, individual engineers can push code without waiting to synchronize with the rest of the team on a big software release. Today on Software Engineering Daily, Flo Motlik, the CTO of Codeship, joins us to discuss continuous integration, dev ops, and microservices. In full disclosure, Codeship is a sponsor of Software Engineering Daily, but we would be doing this interview whether or not that was the case, because Flo has a lot to say about software engineering. In this show, we get into conversations and case studies of how software teams take continuous deployment from theory into practice.

Monsanto is a company that is known for its chemical and biological engineering. It is less well known for its data science and software engineering teams. Tim Williamson is a data scientist at Monsanto, and on today’s show he talked about how he and a small group of engineers at Monsanto dramatically shifted the culture around data science-driven genetic engineering. In this episode, Tim explains how useful graph databases are for modeling the genetic lineages, and talks about how Monsanto manages simulations and experiments on their genomics software pipeline. Tim also talks about how just a few engineers can create a cultural shift within a large company like Monsanto using the leverage allowed by software.

This episode is a republication from my interview with Leslie Lamport on Software Engineering Radio. Leslie Lamport won a Turing Award in 2013 for his work in distributed and concurrent systems. He also designed the document preparation tool LaTex. Leslie is employed by Microsoft Research, and has recently been working with TLA+, a language that is useful for specifying concurrent systems from a high level. The interview begins with a definition: a distributed system is a multiprocessor system in which the time required for interprocess communication is large compared to the time for events within a single processor–in other words, it takes longer for interprocess communication than it does for a process to look at its own memory. Alternatively, a distributed system is one in which processors communicate by sending messages. Leslie goes on to talk about how he became interested in distributed systems, and describes the story behind his paper about the Paxos algorithm. The goal of Paxos is to maintain consensus in an environment with unexpected faults (otherwise known as Byzantine faults). After the discussion of Paxos, Jeff asks Leslie about his recent talk “Thinking for Programmers,” which emphasizes the benefit of having a specification prior to writing actual code. “Specification” can mean a variety of things, but predicates and next-state relationships provide a mathematical rigor that is well-suited to distributed and concurrent systems. The conversation concludes with Jeff asking Leslie about how a programmer can build the mental resolve to work through a difficult problem.

Apache Spark is replacing MATLAB in the domain of computational neuroscience. The constraints of running MATLAB on a single machine can’t support the demands of neuroscience, which has huge collections of images and time-series data sets. Jeremy Freeman is a computational neuroscientist who is adopting Apache Spark to be able to analyze these giant data sets that do not fit on a single machine. But Apache Spark was not designed with neuroscience in mind. For this reason, Jeremy has helped to build several libraries on top of Spark. Thunder is a library for standard, distributed representation of data. Lightning is an API for reproducible web visualizations. These abstractions sit on top of Spark, and add a layer of usability. As it turns out, solving these problems for neuroscience have produced tools that are useful in a variety of other domains. In our discussion with Jeremy Freeman, we talk about Apache Spark, neuroscience, and the technological and cultural problems faced by traditional academic research.

University research produces numerous papers about software engineering. Unfortunately, many of the problems explored by these software engineering researchers have no actual application. These papers fall into The Black Hole of Software Engineering Research. Andy Ko is an associate professor at the University of Washington. When he worked as a CTO of AnswerDash, he experimented with some of the software engineering research coming out of academia and industry. Today on Software Engineering Daily, we discuss Andy’s results. Our conversation spans the topics of education, big companies, and startups.

NewSQL is a class of modern relational databases that seek to provide the same scalable performance of NoSQL systems for OLTP, while still maintaining the ACID guarantees of a traditional database system. VoltDB is a NewSQL database, designed in part by Turing Award Winner Michael Stonebraker. Today’s guest is John Hugg, a founding engineer at VoltDB. Today’s show covers the internals of VoltDB, including concepts like tunable consistency, database replication, and the advantages of a deterministic logical log.

React is a set of technologies started by Facebook and open-sourced in 2013. Since then, it has become widely accepted as the best JavaScript library for building user-interfaces. A dedicated community around React has developed as well, and with any large software community, conferences are a necessity. React.js Conf is a yearly conference in San Francisco where React developers talk about the latest developments in the React ecosystem. React started with user interfaces, but has evolved to be so much more. From data flow patterns like Redux, to hardware libraries and virtual reality–React is no longer just a JavaScript library; it is a philosophy for how to build futuristic technologies today. Brent and Jeff discuss all of this, as well as Exponent, a web development platform for building native mobile apps with JavaScript – using React Native. Brent Vatne is the organizer of React.js Conf and a software engineer at Exponent.

Internet Explorer, Google Chrome, Firefox–it’s easy to forget that these modern browsers descended from the war between Microsoft and Netscape. Today, we hear from a software engineer who was on the front lines of that war, back in 1992. Eric Sink was one of the original developers of Spyglass, a browser that was licensed to both Microsoft AND Netscape. If you want to understand why that happened, and what the inside story of the browser wars is, and what the business of selling browser licenses was like in 1992, check out this episode of Software Engineering Daily.

In this episode, Jeff and Jeremy discuss Internet.org and the criticism it has received from organizations like the EFF. Internet.org is a program created by Facebook whose purpose is to expand the reach of the internet by connecting people in developing nations like India. However, it has come under fire for its failure to provide “the whole internet” and limiting access to a subset of websites which have to be approved by Facebook. Critics call this a violation of net neutrality and argue that it betrays what the internet stands for – openness and fair access to all destinations. This episodes jumps into the nuances of the debate, with Jeff playing counterpoint to the EFF’s stance – that Internet.org is not the internet. Jeremy Malcolm is a technologist and lawyer who works for EFF’s international team on issues such as intellectual property, network neutrality, Internet governance, and trade.

Design is more important than ever. There are so many websites being created every day, that it is not enough to differentiate your product with rock solid engineering. Your product has to be beautiful, intuitive, and tested for usability. You don’t want all of the hard work of the product and engineering teams to go to waste, so on this episode, the focus is design. Tracy Osborn is the creator of WeddingLovely, a beautifully designed two sided marketplace website, where people who are getting married connect with wedding vendors. Today, Tracy discusses “design for nondesigners”. Whether you are an engineer, a marketer, or a business analyst, just learning a few simple tactics about design can take you far.

JavaScript is everywhere, on the browser, the server, and now on hardware. Finally–the Internet of Things is upon us — and it is powered by JavaScript. Even if you consider yourself a web developer, and you have no interest in “maker” culture, keep your eye on this space. Factories in Shenzhen are making better, more modular parts, and the same supply chain economics that are driving down the cost of smart phones and smart cars, will give hobbyist engineers all the simple hardware components they could ever need. This will power a future of proliferated devices that have internet connectivity and “talk” to each other to enable user experiences we wouldn’t dream of today. And naturally, with this will come problems that we cannot anticipate as well. On today’s episode, Andrew discusses how JavaScript can be used on hardware. Jeff and Andrew talk about the players in the hardware space, as well as how the big cloud players like AWS and Azure are positioning themselves to be the data lakes for the internet of things. Andrew is giving a talk about this at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to Fluent, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Andrew Chalkley is a hardware hobbyist, software engineer and a full-time teacher at Treehouse.

Disabled individuals account for over a billion people worldwide. This represents the world’s largest minority on the Web, with $220 billion in discretionary spending power in the United States alone. Yet, the accessible web is more than a financial issue – it is also a moral issue, and it is crucial that we make the web accessible to ensure an equitable information platform. Today’s guest Nic Steenhout joins us to explain how to develop software with accessibility in mind. At the upcoming Fluent Conference, he will be giving a talk entitled Don’t turn off that JavaScript just yet, referencing JavaScript’s history of causing accessibility problems. If you want to win a free ticket to Fluent, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Nic Steenhout is a software engineer and an accessibility consultant at Simply Accessible.

Reactive programming emphasizes writing code that is readily responsive to events. It is an increasingly popular paradigm with highly interactive websites like Netflix. It draws on the value of functional programming calls like map, reduce, and flatmap. Matt joins us today to chat about reactive programming, the observer pattern, and his thoughts on where the future is going. If you are a fan of functional programming or JavaScript, you will enjoy this show. Matt is also speaking at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to the conference, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Matt Podwysocki is the author ReactiveX JS and a software engineer at Microsoft.

Node.js powers an increasing number of applications in the modern web. As node’s popularity grew, npm evolved in parallel as its default package manager and it has become a robust system for sharing and developing node programs. Yet today, npm is growing beyond its roots, and poised to become a generalized framework for all kinds of workflows within web development. Laurie will be giving a talk at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to the conference, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Laurie Voss is the CTO of npm, Inc.

Following the successful experiment of History of Hadoop, we are doing another Saturday experiment: an editorial podcast. Let us know your thoughts via Slack, Twitter, or email! Our podcast errs on the side of technical rigor. Whether the topic is distributed databases, microservices, Soylent, Uber, or Dwarf Fortress, we try to separate hype from substance, deferring the narrative to the guest. With that deference there is editorial objectivity on the part of Software Engineering Daily. One Slack channel member said “when Software Engineering Daily’s opinions are announced, that takes time away from the guest.” Fair enough. However, as with any journalistic organization, we have opinions. On SE Daily, we stream objectivity and batch subjectivity. This episode is willfully subjective. It has no guest. It is a monologue editorial inspired by Developer Tea. Immutable laws are rare in software engineering, and when an engineer claims to have found one, that engineer is usually regarded with skepticism. General principles are more welcome. In this post and podcast episode, I convey some loose philosophies about modern software engineering. These are strong opinions weakly held. I welcome debate and discussion. Software is a new field and nobody knows how to do it. If someone says you are unqualified and therefore you must do maintenance work, you should question that person. We have an upside down system where the people who are paid the least do the crappiest work. They tend to be young and naive. This is not an axiom. The narrative that is sold to young engineers by giant companies is the following: take your $80k/yr job, do the software maintenance which makes the company $1 million, and hate your life. After you have spent enough time in the first tier of the intellectual strip mine, we will make you an SDE 2, where you can do slightly higher level refactoring for $150k a year, which will make the giant company $5 million. This is what is called an arbitrage. We have an assembly line mindset left over from the industrial age. Software is more of an artisanship. Don’t believe that you are a replaceable cog. Don’t believe the one-size-fits-all interview process with whiteboarding problems. These serve to grind away your individuality and make you feel like an assembly line worker. The planning and design process is an art, but once the requirements are in place you can proceed more deterministically. The same is true for the other quantitative activities I have taken part in–poker, music, and writing. As Michael Rosenthal and I discussed, the question of art vs. science is the same question as strategy vs. tactics. I learned this very early on playing poker when I had to leave that career, and I had tightly coupled poker to my identity. If you make your job the same thing as who you are, then your self-worth is defined by those who are judging you in your job. Your job is a means to service your own higher purpose. When you take an action on your smartphone, there is latency before that action is ingested. Servers sometimes will lie to you, but servers tend towards eventual consistency. The world works the same way. In the short term, human systems lie to us all the time, but the world tends toward eventual consistency–the truth eventually presents itself. A decent analogy is the efficient market hypothesis: slowly efficient markets are an eventually consistent process. The world is a distributed system–what is the consequence of this? We have to do the arduous risk and reward calculations that are mandatory for every distributed systems programmer. Long-tail failures can and do occur. In a distributed system, we often prioritize safety over liveness. In a distributed operating system, the programmer takes all precaution to avoid data loss. Similarly, if a real life scenario presents a small probability of giant downside risk, you should take huge precaution. If someone offers you to roll a die with 1000...

Lin Clark is today’s guest on Software Engineering Daily, and she joins Jeff to talk about Code Cartoons, a webcomic that explains Facebook’s open source projects like Flux and Relay with the same elegance and creativity as XKCD. Lin explains why cartoons can be a surprisingly effective way to introduce technologies to people without the fuss and intimidation of written tutorials or documentation. At the 2016 O’Reilly Fluent Conference, Lin Clark will be speaking about Code Cartoons and the technologies that have sprung up around React JS. Software Engineering Daily is giving away a free ticket to the Fluent 2016 conference, which takes place on March 8-10 in San Francisco. If you want to be entered to win this free ticket, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Lin Clark is the creator of Code Cartoons, and a Senior Developer Tools Engineer at Mozilla.