Software Engineering Daily

In today’s episode, Ben Halpern interviews Juan Buritica, VP of Engineering at Ride. They discuss Ride’s migration from a monolith to microservices, the challenges of running a distributed team and preventing developer burnout.

GreyOrange Robotics builds robots for warehouse automation of logistics and ecommerce companies for quicker deliveries. Today’s episode features Akash Gupta, the CTO of GreyOrange.

DevOps has been the subject of many episodes of Software Engineering Daily. And yet–the question we continue to ask ourselves is “what is DevOps?” In order to understand the present, we must look into the past. On today’s show, we break down the history of dev ops with Demir Selmanovic, the lead technical editor at TopTal. From waterfall software development, to the agile manifesto. From lean manufacturing to the productive pinnacle of DevOps nirvana. Demir also wrote a blog post called What the Hell is DevOps, which he posted on the TopTal blog. Full disclosure: TopTal is a sponsor of our show, but this episode doesn’t discuss TopTal very much so hopefully that doesn’t impact the trust of our listeners.

Today’s guest is Calvin French-Owen, the CTO of Segment, a tool that companies use to aggregate their analytics into once place. As Segment has scaled, the company has had to restructure its etire technical architecture. Microservices, containers, Amazon Web Services, and dev ops are a few of the topics that Calvin and I explore in our conversation, so this is a great episode for anyone who is trying to understand the relationships between those different subjects. Segment’s product unifies analytics from different services and puts them into one centralized place. Full disclosure: Segment is a sponsor of Software Engineering Daily. For most of this episode, we don’t even talk about the product, we talk about the back end engineering behind the product.

CodeNewbie is a community of programmers and people learning to code. There are so many people learning about software today, and CodeNewbie gives them a place to hang out, socialize, and become comfortable with the world of software. CodeNewbie has an excellent podcast, and if you like Software Engineering Daily you should check it out. Saron Yitbarek is the guest today, she is the creator of CodeNewbie. In this episode of Software Engineering Daily, Saron sits down to talk about learning to code after she had already had several careers, spanning biology, public radio, and marketing.

Mic.com is a media company focused on news for millennials. Anthony Sessa is the VP of product at Mic.com, and he joins us to talk about the engineering of a modern news organization tailored to young people. We discuss data engineering, frontend technologies, and how to migrate away from WordPress. We also explore how to build a successful media company today, and how millennials want to consume news and software.

Software developers have been socializing on chat rooms for decades. In the nineties, we began using IRC and AOL instant messenger. In the early 2000s, we turned to Google Hangouts and Yammer. Today, we are using modern apps like Slack and Hipchat. On today’s show, we take a deep dive into Gitter, the chat client specifically designed for developers. Our guests are Mike Bartlett and Andrew Newdigate, the creators of Gitter. Gitter is a highly scalable, real time social application for developers to talk about writing their software. This is a great episode that spans topics like front end development, back end distributed systems, how to compete with Slack, and how to scale a chat room to tens of thousands of active users.

Military veterans have the right set of skills to become programmers. Technical expertise, emotional resilience, psychological persistence, and teamwork–these are the qualities of the US army and they are the qualities of the best programmers. More and more veterans who leave the army are becoming coders, and the mission of Operation Code is to help veterans make that transition. Dave Molina is the founder of Operation Code. His organization helps army veterans become coders. Dave came on the show to discuss coding boot camps, military, software education, and what it’s like to become a civilian technologist after spending years in the army.

Building a software business today requires lots of decision making. Building software isn’t just about choosing a programming language, or a framework, or a database. Developers have to choose the right cloud service provider, the best issue tracking service, the best hosted code repository, the clearest data visualization tools. We need a platform to compare the options between different products, all in one place. In today’s episode, we talk to Ayan Barua from Siftery, a platform for software products and the companies who use them. Siftery’s mission is to help technologists find the best products for their job, whether they are a sysadmin or an advertising account manager. My discussion with Ayan covers the question of build versus buy, how to build a recommender system, and how software will be built in the future; this is also a great episode for anyone who is thinking of how to build a two-sided marketplace business.

Software engineering is a deterministic field. We write lines of code, and feed data into that code, expecting to get a certain answer. Computing is deterministic because humans developed it–we understand computers from top to bottom. The same cannot be said about biology. Matt Might is an associate professor with a PhD in computer science. When his son was diagnosed with an extremely rare illness, he was confronted with the uncertainties of human biology. In this episode, we discuss Matt’s quest to solve the puzzle of his son’s disease–computer science meets genetics, on this episode of Software Engineering Daily.

In many ways the engineering job market is broken. Engineers cannot find the right jobs where they are valued properly and enjoy their work. Employers have difficulty filling their positions and finding the right people that fit their cultures. Traditional matchmakers like Monster and generic recruiting agencies don’t cut it — you know this by the generic LinkedIn messages you get looking for a Java expert… when you have JavaScript as a skill on your profile. Hired.com is trying to fix this broken process by using a high-touch approach to recruiting. Hiten Parmar, an engineer and product strategist at Hired joins us today to discuss the issues at hand, as well as how Hired is approaching the solution. We chat about the challenges facing engineers, which skills are in demand, and the inner workings of Hired. Full disclosure: Hired is a sponsor of the show, but we would still have done this interview because we genuinely want to discuss the job market, and figure out how companies are addressing this issue.

Stack Overflow is used by developers to find out how to build software. Stack Overflow is both a tool and a community, and today’s guest Jeff Atwood has made a career out of building tools and communities. As the co-founder of Stack Exchange and Discourse.org, Jeff has been solving the problem of civilized online communication for seven years. In today’s episode of Software Engineering Daily, Jeff Atwood talks about building online communities from the perspective of an engineer as well as a sociologist.

On Software Engineering Daily, we often discuss big data in terms of data engineering and data science. Data engineering is the infrastructure and pipelines that handle massive amounts of data and puts that data in a data lake. With a data infrastructure in place, a data scientist can study the data and take action on it. A data scientist can also create visualizations–which is the subject of today’s episode. Aurelia Moser has a specialty in mapping and data visualization. We discussed how to create effective visualizations of our data, and the tools that can be used to collect and present our data. Whether you work at the New York Times or a small tech company, data visualizations are important, because visualizations can be used to communicate important trends across an organization. Aurelia will also be speaking at the upcoming Strata + Hadoop World Conference in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Big data is yesterday–fast data is now. FiloDB is a reactive columnar OLAP database that is built on Cassandra and Spark. Today’s guest is Evan Chan, creator of FiloDB. In our discussion today, we talk about the use cases of an OLAP data store. Evan explains how to tackle the problem of video analytics–if you have ever found yourself asking how a company like YouTube or Netflix or Ooyala performs analytics on millions of users watching millions of videos, this episode is for you. By combining the database features of Cassandra with the data processing power of Spark, Evan created FiloDB to help solve this type of analytics problem. Evan will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Apache Cassandra can serve as both the real-time data store for online transactional applications, as well as the read-intensive database for data warehousing operations. In order to combine these two use cases into a single database, Apache Cassandra required lots of innovation. In today’s episode of Software Engineering Daily, we discuss the internals of Cassandra. Tim Berglund takes us through how Cassandra performs reads and writes, how Cassandra offers tunable consistency, and what inspirations Cassandra took from the Amazon Dynamo paper as well as the Google BigTable paper. Tim will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Hadoop was created in 2003. In the early years, Hadoop provided large scale data processing with MapReduce, and distributed fault-tolerant storage with the Hadoop Distributed File System. Over the last decade, Hadoop has evolved rapidly, with the support of a big open-source community. Today’s guest is Mike Cafarella, co-creator of Hadoop. Mike takes us on a journey from past to present. Hadoop was based on the Google File System and MapReduce papers, and so Mike and I talk about what it was like to work on a distributed file system in 2004, and the challenges of implementing real software systems based on white papers. We also discuss YARN, and the wave of innovation that YARN enabled within the Hadoop ecosystem. Mike will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

When a company gets big enough, there is so much data to be processed that an entire data engineering team becomes responsible for managing this data and making it available to other teams. Airbnb is one such company. Max Beauchemin works on the data engineering team at Airbnb, where he creates infrastructure and tooling for managing data. In this episode of Software Engineering Daily, we talk about AIrflow, a workflow scheduler that assists in job processing. If you don’t know what a workflow is, or a job, we will explain that in this episode. Max and I also talk about Panoramix, a data slicing and visualization tool that helps data scientists and business analysts understand large volumes of data. Max will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Diagnosing illness today requires the trained eye of a doctor. With machine learning, we might someday be able to diagnose illness using only a data set. Today on Software Engineering Daily, we are joined by David Kale, a researcher at the intersection of machine learning and clinical data. We discuss the machine learning and research techniques he is using to diagnose illnesses using neural networks, and we also talk about the challenges of performing data science in hospitals, where the data is mostly confidential. David will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.

Ruby on Rails has had the most commits made by its creator, David Heinemeier Hansson. The next most frequent contributor to Rails is Aaron Patterson, our guest on today’s episode of Software Engineering Daily. Topics discussed in today’s episode include Ruby relative to JavaScript, Rails relative to Linux, and DHH relative to Tenderlove.

Microsoft and Amazon are the tech giants of the Pacific Northwest. These two companies shape Seattle technology, and Todd Bishop has been writing about them for over a decade. Todd is the co-founder of GeekWire, an online media company focused on technology in the Pacific Northwest. We talk about the past and present of Microsoft and Amazon, what it means for an engineering company to be good or evil, and what it’s like to be a software journalist.

Socket.io enables realtime bidirectional communication. But what does “realtime” actually mean? Today’s guest is Guillermo Rauch, the creator of Socket.IO, a widely used technology for client server communication. We discuss the nature of real-time apps like Uber and Google Docs, and talk about the API and usage of Socket.IO.

Technology careers are not as accessible to certain demographics, such as women. Sarah Allen is working to change that through education and outreach. Sarah Allen created RailsBridge and Bridge Foundry, a pair of organizations working to make technology more accessible to people who are underrepresented in tech. RailsBridge workshops are a fun and free way to get started with Ruby, Rails or other technologies. RailsBridge was so successful that Sarah created Bridge Foundry, which has an even wider variety of free workshops, from Clojure to Go to Mobile. This is an inspirational episode of Software Engineering Daily, and is a great learning opportunity for anyone interested in building a community.

Continuous integration and deployment are important tools for modern software development. With continuous integration and deployment, individual engineers can push code without waiting to synchronize with the rest of the team on a big software release. Today on Software Engineering Daily, Flo Motlik, the CTO of Codeship, joins us to discuss continuous integration, dev ops, and microservices. In full disclosure, Codeship is a sponsor of Software Engineering Daily, but we would be doing this interview whether or not that was the case, because Flo has a lot to say about software engineering. In this show, we get into conversations and case studies of how software teams take continuous deployment from theory into practice.

Monsanto is a company that is known for its chemical and biological engineering. It is less well known for its data science and software engineering teams. Tim Williamson is a data scientist at Monsanto, and on today’s show he talked about how he and a small group of engineers at Monsanto dramatically shifted the culture around data science-driven genetic engineering. In this episode, Tim explains how useful graph databases are for modeling the genetic lineages, and talks about how Monsanto manages simulations and experiments on their genomics software pipeline. Tim also talks about how just a few engineers can create a cultural shift within a large company like Monsanto using the leverage allowed by software.

This episode is a republication from my interview with Leslie Lamport on Software Engineering Radio. Leslie Lamport won a Turing Award in 2013 for his work in distributed and concurrent systems. He also designed the document preparation tool LaTex. Leslie is employed by Microsoft Research, and has recently been working with TLA+, a language that is useful for specifying concurrent systems from a high level. The interview begins with a definition: a distributed system is a multiprocessor system in which the time required for interprocess communication is large compared to the time for events within a single processor–in other words, it takes longer for interprocess communication than it does for a process to look at its own memory. Alternatively, a distributed system is one in which processors communicate by sending messages. Leslie goes on to talk about how he became interested in distributed systems, and describes the story behind his paper about the Paxos algorithm. The goal of Paxos is to maintain consensus in an environment with unexpected faults (otherwise known as Byzantine faults). After the discussion of Paxos, Jeff asks Leslie about his recent talk “Thinking for Programmers,” which emphasizes the benefit of having a specification prior to writing actual code. “Specification” can mean a variety of things, but predicates and next-state relationships provide a mathematical rigor that is well-suited to distributed and concurrent systems. The conversation concludes with Jeff asking Leslie about how a programmer can build the mental resolve to work through a difficult problem.

Apache Spark is replacing MATLAB in the domain of computational neuroscience. The constraints of running MATLAB on a single machine can’t support the demands of neuroscience, which has huge collections of images and time-series data sets. Jeremy Freeman is a computational neuroscientist who is adopting Apache Spark to be able to analyze these giant data sets that do not fit on a single machine. But Apache Spark was not designed with neuroscience in mind. For this reason, Jeremy has helped to build several libraries on top of Spark. Thunder is a library for standard, distributed representation of data. Lightning is an API for reproducible web visualizations. These abstractions sit on top of Spark, and add a layer of usability. As it turns out, solving these problems for neuroscience have produced tools that are useful in a variety of other domains. In our discussion with Jeremy Freeman, we talk about Apache Spark, neuroscience, and the technological and cultural problems faced by traditional academic research.

University research produces numerous papers about software engineering. Unfortunately, many of the problems explored by these software engineering researchers have no actual application. These papers fall into The Black Hole of Software Engineering Research. Andy Ko is an associate professor at the University of Washington. When he worked as a CTO of AnswerDash, he experimented with some of the software engineering research coming out of academia and industry. Today on Software Engineering Daily, we discuss Andy’s results. Our conversation spans the topics of education, big companies, and startups.

NewSQL is a class of modern relational databases that seek to provide the same scalable performance of NoSQL systems for OLTP, while still maintaining the ACID guarantees of a traditional database system. VoltDB is a NewSQL database, designed in part by Turing Award Winner Michael Stonebraker. Today’s guest is John Hugg, a founding engineer at VoltDB. Today’s show covers the internals of VoltDB, including concepts like tunable consistency, database replication, and the advantages of a deterministic logical log.

React is a set of technologies started by Facebook and open-sourced in 2013. Since then, it has become widely accepted as the best JavaScript library for building user-interfaces. A dedicated community around React has developed as well, and with any large software community, conferences are a necessity. React.js Conf is a yearly conference in San Francisco where React developers talk about the latest developments in the React ecosystem. React started with user interfaces, but has evolved to be so much more. From data flow patterns like Redux, to hardware libraries and virtual reality–React is no longer just a JavaScript library; it is a philosophy for how to build futuristic technologies today. Brent and Jeff discuss all of this, as well as Exponent, a web development platform for building native mobile apps with JavaScript – using React Native. Brent Vatne is the organizer of React.js Conf and a software engineer at Exponent.

Internet Explorer, Google Chrome, Firefox–it’s easy to forget that these modern browsers descended from the war between Microsoft and Netscape. Today, we hear from a software engineer who was on the front lines of that war, back in 1992. Eric Sink was one of the original developers of Spyglass, a browser that was licensed to both Microsoft AND Netscape. If you want to understand why that happened, and what the inside story of the browser wars is, and what the business of selling browser licenses was like in 1992, check out this episode of Software Engineering Daily.

In this episode, Jeff and Jeremy discuss Internet.org and the criticism it has received from organizations like the EFF. Internet.org is a program created by Facebook whose purpose is to expand the reach of the internet by connecting people in developing nations like India. However, it has come under fire for its failure to provide “the whole internet” and limiting access to a subset of websites which have to be approved by Facebook. Critics call this a violation of net neutrality and argue that it betrays what the internet stands for – openness and fair access to all destinations. This episodes jumps into the nuances of the debate, with Jeff playing counterpoint to the EFF’s stance – that Internet.org is not the internet. Jeremy Malcolm is a technologist and lawyer who works for EFF’s international team on issues such as intellectual property, network neutrality, Internet governance, and trade.

Design is more important than ever. There are so many websites being created every day, that it is not enough to differentiate your product with rock solid engineering. Your product has to be beautiful, intuitive, and tested for usability. You don’t want all of the hard work of the product and engineering teams to go to waste, so on this episode, the focus is design. Tracy Osborn is the creator of WeddingLovely, a beautifully designed two sided marketplace website, where people who are getting married connect with wedding vendors. Today, Tracy discusses “design for nondesigners”. Whether you are an engineer, a marketer, or a business analyst, just learning a few simple tactics about design can take you far.

JavaScript is everywhere, on the browser, the server, and now on hardware. Finally–the Internet of Things is upon us — and it is powered by JavaScript. Even if you consider yourself a web developer, and you have no interest in “maker” culture, keep your eye on this space. Factories in Shenzhen are making better, more modular parts, and the same supply chain economics that are driving down the cost of smart phones and smart cars, will give hobbyist engineers all the simple hardware components they could ever need. This will power a future of proliferated devices that have internet connectivity and “talk” to each other to enable user experiences we wouldn’t dream of today. And naturally, with this will come problems that we cannot anticipate as well. On today’s episode, Andrew discusses how JavaScript can be used on hardware. Jeff and Andrew talk about the players in the hardware space, as well as how the big cloud players like AWS and Azure are positioning themselves to be the data lakes for the internet of things. Andrew is giving a talk about this at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to Fluent, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Andrew Chalkley is a hardware hobbyist, software engineer and a full-time teacher at Treehouse.

Disabled individuals account for over a billion people worldwide. This represents the world’s largest minority on the Web, with $220 billion in discretionary spending power in the United States alone. Yet, the accessible web is more than a financial issue – it is also a moral issue, and it is crucial that we make the web accessible to ensure an equitable information platform. Today’s guest Nic Steenhout joins us to explain how to develop software with accessibility in mind. At the upcoming Fluent Conference, he will be giving a talk entitled Don’t turn off that JavaScript just yet, referencing JavaScript’s history of causing accessibility problems. If you want to win a free ticket to Fluent, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Nic Steenhout is a software engineer and an accessibility consultant at Simply Accessible.

Reactive programming emphasizes writing code that is readily responsive to events. It is an increasingly popular paradigm with highly interactive websites like Netflix. It draws on the value of functional programming calls like map, reduce, and flatmap. Matt joins us today to chat about reactive programming, the observer pattern, and his thoughts on where the future is going. If you are a fan of functional programming or JavaScript, you will enjoy this show. Matt is also speaking at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to the conference, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Matt Podwysocki is the author ReactiveX JS and a software engineer at Microsoft.

Node.js powers an increasing number of applications in the modern web. As node’s popularity grew, npm evolved in parallel as its default package manager and it has become a robust system for sharing and developing node programs. Yet today, npm is growing beyond its roots, and poised to become a generalized framework for all kinds of workflows within web development. Laurie will be giving a talk at the 2016 O’Reilly Fluent Conference. If you want to win a free ticket to the conference, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Laurie Voss is the CTO of npm, Inc.

Following the successful experiment of History of Hadoop, we are doing another Saturday experiment: an editorial podcast. Let us know your thoughts via Slack, Twitter, or email! Our podcast errs on the side of technical rigor. Whether the topic is distributed databases, microservices, Soylent, Uber, or Dwarf Fortress, we try to separate hype from substance, deferring the narrative to the guest. With that deference there is editorial objectivity on the part of Software Engineering Daily. One Slack channel member said “when Software Engineering Daily’s opinions are announced, that takes time away from the guest.” Fair enough. However, as with any journalistic organization, we have opinions. On SE Daily, we stream objectivity and batch subjectivity. This episode is willfully subjective. It has no guest. It is a monologue editorial inspired by Developer Tea. Immutable laws are rare in software engineering, and when an engineer claims to have found one, that engineer is usually regarded with skepticism. General principles are more welcome. In this post and podcast episode, I convey some loose philosophies about modern software engineering. These are strong opinions weakly held. I welcome debate and discussion. Software is a new field and nobody knows how to do it. If someone says you are unqualified and therefore you must do maintenance work, you should question that person. We have an upside down system where the people who are paid the least do the crappiest work. They tend to be young and naive. This is not an axiom. The narrative that is sold to young engineers by giant companies is the following: take your $80k/yr job, do the software maintenance which makes the company $1 million, and hate your life. After you have spent enough time in the first tier of the intellectual strip mine, we will make you an SDE 2, where you can do slightly higher level refactoring for $150k a year, which will make the giant company $5 million. This is what is called an arbitrage. We have an assembly line mindset left over from the industrial age. Software is more of an artisanship. Don’t believe that you are a replaceable cog. Don’t believe the one-size-fits-all interview process with whiteboarding problems. These serve to grind away your individuality and make you feel like an assembly line worker. The planning and design process is an art, but once the requirements are in place you can proceed more deterministically. The same is true for the other quantitative activities I have taken part in–poker, music, and writing. As Michael Rosenthal and I discussed, the question of art vs. science is the same question as strategy vs. tactics. I learned this very early on playing poker when I had to leave that career, and I had tightly coupled poker to my identity. If you make your job the same thing as who you are, then your self-worth is defined by those who are judging you in your job. Your job is a means to service your own higher purpose. When you take an action on your smartphone, there is latency before that action is ingested. Servers sometimes will lie to you, but servers tend towards eventual consistency. The world works the same way. In the short term, human systems lie to us all the time, but the world tends toward eventual consistency–the truth eventually presents itself. A decent analogy is the efficient market hypothesis: slowly efficient markets are an eventually consistent process. The world is a distributed system–what is the consequence of this? We have to do the arduous risk and reward calculations that are mandatory for every distributed systems programmer. Long-tail failures can and do occur. In a distributed system, we often prioritize safety over liveness. In a distributed operating system, the programmer takes all precaution to avoid data loss. Similarly, if a real life scenario presents a small probability of giant downside risk, you should take huge precaution. If someone offers you to roll a die with 1000...

Lin Clark is today’s guest on Software Engineering Daily, and she joins Jeff to talk about Code Cartoons, a webcomic that explains Facebook’s open source projects like Flux and Relay with the same elegance and creativity as XKCD. Lin explains why cartoons can be a surprisingly effective way to introduce technologies to people without the fuss and intimidation of written tutorials or documentation. At the 2016 O’Reilly Fluent Conference, Lin Clark will be speaking about Code Cartoons and the technologies that have sprung up around React JS. Software Engineering Daily is giving away a free ticket to the Fluent 2016 conference, which takes place on March 8-10 in San Francisco. If you want to be entered to win this free ticket, tweet about your favorite episode of Software Engineering Daily and tag @fluentconf and @software_daily. Lin Clark is the creator of Code Cartoons, and a Senior Developer Tools Engineer at Mozilla.

Prepare for the post-browser world, says today’s guest Jonathan Stark. We are moving beyond the browser, into a world where the web is consumed by a variety of front ends. Jonathan is a speaker at the upcoming O’Reilly Fluent Conference in San Francisco, and you can win a free ticket to the conference. To be entered into a random drawing for that ticket, send us a tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. Fluent is a conference about the web platform–which certainly includes the browser, but also mobile apps, VR experiences, and smart objects connected to the web. In today’s episode, Jeff and Jonathan explore what adjustments developers should make in this post-browser world. Jonathan Stark is a mobile consultant who helps consumer brands thrive in the post-PC era.

You keep hearing about containers, and maybe you have even used Docker in production. Now, it’s time to move beyond Docker. In today’s episode of Software Engineering Daily, Steve Pousty talks about OpenShift, a platform from RedHat that helps engineers leverage the power of containers and the dev ops harmony of microservices. To hear these buzzwords, you can always listen to Software Engineering Daily. But to see these buzzwords demonstrated in live presentations, the 2016 O’Reilly Fluent Conference is coming up March 8-10 in San Francisco, and you can get a chance to win a free ticket by tweeting at us about your favorite episode of Software Engineering Daily. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. Steve Pousty is a Developer Advocate (and PaaS Dust Spreader) at Red Hat.

In today’s episode of SEDaily, Jeff and Austin discuss why designing for a drunk user is a competitive advantage. Austin also explains why engineers need to think about UX and how they can incorporate UX considerations into development workflows. Austin Knight is a designer and UX engineer at Hubspot. Austin will be speaking at the upcoming Fluent Conference, and Software Engineering Daily is giving away a ticket to Fluent, which will be held March 8-10 in San Francisco. To be entered in a random drawing for this free ticket, tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Make sure to tag us @software_daily and Fluent @fluentconf so we see your tweet.

This week of shows on Software Engineering Daily focuses on O’Reilly’s Fluent Conference, a yearly software conference that focuses on the web and the tools used to build modern applications. Software Engineering Daily is giving away a ticket to The Fluent Conference, which will be held March 8-10 in San Francisco. To be entered in a random drawing for this free ticket, tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. In today’s show, Jeff chats with Peter Cooper and Simon St Laurent, the organizers of Fluent. They discuss how the web is transitioning beyond the desktop and how to keep up with ever evolving javascript frameworks. This was also unique chance as well to learn why software conferences are so important.

This episode is different from the traditional interview format of Software Engineering Daily, and focuses on the history of Hadoop. Thanks to Marco Bonaci for allowing us to republish this in audio format. You can find the original post here: History of Hadoop

Operating a data center has been turned into a service. For most new companies, managing real-world servers is no longer an issue. But one exception is if your startup is a company offering the data center as a service. In this episode, Jeff and Sam talk about the engineering that goes on at DigitalOcean, and how the company differentiates itself from its competitors. Sam Kottler is an engineering manager at DigitalOcean, and previously worked at Red Hat.

Monoliths versus microservices – the architectural debate continues across the internet. SoundCloud is a popular music company that experienced the rapid development benefits of a monolith, as well as the long-term technical debt. That technical debt has since been relieved by a move towards microservices. In this episode of Software Engineering Daily, Lukasz joins Jeff to talk about the realities of moving from a monolith to a microservices architecture, and walks through the lessons learned at SoundCloud. Lukasz Plotnicki is a consultant and software engineer at ThoughtWorks, which helps clients address their software development challenges and needs.

Stream processing engines are often the component of data engineering stacks with the most variety, with the big data ecosystem offering several compelling options. Assessing the differences between projects like Flink, Storm, and Spark Streaming is difficult without an agreed upon set of metrics to compare them on. Fortunately, the Yahoo engineering team created a set of benchmarks to do exactly this. On today’s episode, Jeff and Bobby compare streaming frameworks through the performance on several of Yahoo’s benchmarking tests. Bobby Evans is an architect at Yahoo working on streaming frameworks, primarily on Apache Storm. He is also the Apache Storm Incubating PMC at The Apache Software Foundation.

Software engineering combines art, science, and philosophy. On the Developer Tea podcast, Jonathan Cutrell explores how engineers can improve their technical ability as well as their interpersonal skills and mental clarity. On this episode of Software Engineering Daily, Jeff and Jonathan about focus, career development, and software podcasting. Jonathan Cutrell is the founder and host of Developer Tea. He is also the Director of Technology at Whiteboard.

Matplotlib is a python plotting library inspired by MATLAB. It allows developers to easily visualize their data using simple functions. In this episode, Jeff and Ben discuss why visualizing data is integral for the human experience, and how they it can make drawing conclusions from data more meaningful. Ben relates this to his experience studying atmospheric data in the wild, and how his work as a researcher drives his need to visualize the vast amounts of raw data generated by weather systems. Ben Root is a core contributor to Matplotlib and works as a scientific programmer for Atmospheric and Environmental Research Inc.

Keras is a minimalist, highly modular neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. In this episode, François discusses the state of deep learning, and explains why the field is experiencing a cambrian explosion that eventually may taper off. He explains the need for Keras and why its simplicity and ease makes it a useful deep learning library for developers to experiment and build with. François Chollet is the author of Keras and the founder of Wysp, learning platform for artists. He currently works for Google as a deep learning engineer and researcher.

Software is changing every domain including Marketing, Healthcare, and even Agriculture. Government is one specific area where the potential for improvement is obvious. Governments are often defined by abstract rules, transactions, permissions and hierarchies, grounded on some fundamental principles – this is striking similar to the same tenets that software and computer science are based on. In that sense, could software be leveraged in this unique industry to create massive change? OpenGov is one company trying to find out through application of software in local city and state governments. The company currently offers a web-based financial visualization software that focuses on making government budget and financial data accessible by modeling and presenting it in a user-friendly way. But it is not farfetched to see the potential being greater and broader in scope. In this episode, Jeff discussed with Andrew how OpenGov is using technology to improve government. Their conversation focuses in on the front-end technologies used to create the webapp, as well as “microservice fatigue” and the other various architectural concerns within the team. Andrew Clark is a software engineer at OpenGov.