Software Engineering Daily

Prepare for the post-browser world, says today’s guest Jonathan Stark. We are moving beyond the browser, into a world where the web is consumed by a variety of front ends. Jonathan is a speaker at the upcoming O’Reilly Fluent Conference in San Francisco, and you can win a free ticket to the conference. To be entered into a random drawing for that ticket, send us a tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. Fluent is a conference about the web platform–which certainly includes the browser, but also mobile apps, VR experiences, and smart objects connected to the web. In today’s episode, Jeff and Jonathan explore what adjustments developers should make in this post-browser world. Jonathan Stark is a mobile consultant who helps consumer brands thrive in the post-PC era.

You keep hearing about containers, and maybe you have even used Docker in production. Now, it’s time to move beyond Docker. In today’s episode of Software Engineering Daily, Steve Pousty talks about OpenShift, a platform from RedHat that helps engineers leverage the power of containers and the dev ops harmony of microservices. To hear these buzzwords, you can always listen to Software Engineering Daily. But to see these buzzwords demonstrated in live presentations, the 2016 O’Reilly Fluent Conference is coming up March 8-10 in San Francisco, and you can get a chance to win a free ticket by tweeting at us about your favorite episode of Software Engineering Daily. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. Steve Pousty is a Developer Advocate (and PaaS Dust Spreader) at Red Hat.

In today’s episode of SEDaily, Jeff and Austin discuss why designing for a drunk user is a competitive advantage. Austin also explains why engineers need to think about UX and how they can incorporate UX considerations into development workflows. Austin Knight is a designer and UX engineer at Hubspot. Austin will be speaking at the upcoming Fluent Conference, and Software Engineering Daily is giving away a ticket to Fluent, which will be held March 8-10 in San Francisco. To be entered in a random drawing for this free ticket, tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Make sure to tag us @software_daily and Fluent @fluentconf so we see your tweet.

This week of shows on Software Engineering Daily focuses on O’Reilly’s Fluent Conference, a yearly software conference that focuses on the web and the tools used to build modern applications. Software Engineering Daily is giving away a ticket to The Fluent Conference, which will be held March 8-10 in San Francisco. To be entered in a random drawing for this free ticket, tweet about your favorite episode of Software Engineering Daily between now and February 22nd. Include the hashtag #fluentconf and and tag us @software_daily to make sure we can see your tweet. In today’s show, Jeff chats with Peter Cooper and Simon St Laurent, the organizers of Fluent. They discuss how the web is transitioning beyond the desktop and how to keep up with ever evolving javascript frameworks. This was also unique chance as well to learn why software conferences are so important.

This episode is different from the traditional interview format of Software Engineering Daily, and focuses on the history of Hadoop. Thanks to Marco Bonaci for allowing us to republish this in audio format. You can find the original post here: History of Hadoop

Operating a data center has been turned into a service. For most new companies, managing real-world servers is no longer an issue. But one exception is if your startup is a company offering the data center as a service. In this episode, Jeff and Sam talk about the engineering that goes on at DigitalOcean, and how the company differentiates itself from its competitors. Sam Kottler is an engineering manager at DigitalOcean, and previously worked at Red Hat.

Monoliths versus microservices – the architectural debate continues across the internet. SoundCloud is a popular music company that experienced the rapid development benefits of a monolith, as well as the long-term technical debt. That technical debt has since been relieved by a move towards microservices. In this episode of Software Engineering Daily, Lukasz joins Jeff to talk about the realities of moving from a monolith to a microservices architecture, and walks through the lessons learned at SoundCloud. Lukasz Plotnicki is a consultant and software engineer at ThoughtWorks, which helps clients address their software development challenges and needs.

Stream processing engines are often the component of data engineering stacks with the most variety, with the big data ecosystem offering several compelling options. Assessing the differences between projects like Flink, Storm, and Spark Streaming is difficult without an agreed upon set of metrics to compare them on. Fortunately, the Yahoo engineering team created a set of benchmarks to do exactly this. On today’s episode, Jeff and Bobby compare streaming frameworks through the performance on several of Yahoo’s benchmarking tests. Bobby Evans is an architect at Yahoo working on streaming frameworks, primarily on Apache Storm. He is also the Apache Storm Incubating PMC at The Apache Software Foundation.

Software engineering combines art, science, and philosophy. On the Developer Tea podcast, Jonathan Cutrell explores how engineers can improve their technical ability as well as their interpersonal skills and mental clarity. On this episode of Software Engineering Daily, Jeff and Jonathan about focus, career development, and software podcasting. Jonathan Cutrell is the founder and host of Developer Tea. He is also the Director of Technology at Whiteboard.

Matplotlib is a python plotting library inspired by MATLAB. It allows developers to easily visualize their data using simple functions. In this episode, Jeff and Ben discuss why visualizing data is integral for the human experience, and how they it can make drawing conclusions from data more meaningful. Ben relates this to his experience studying atmospheric data in the wild, and how his work as a researcher drives his need to visualize the vast amounts of raw data generated by weather systems. Ben Root is a core contributor to Matplotlib and works as a scientific programmer for Atmospheric and Environmental Research Inc.

Keras is a minimalist, highly modular neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. In this episode, François discusses the state of deep learning, and explains why the field is experiencing a cambrian explosion that eventually may taper off. He explains the need for Keras and why its simplicity and ease makes it a useful deep learning library for developers to experiment and build with. François Chollet is the author of Keras and the founder of Wysp, learning platform for artists. He currently works for Google as a deep learning engineer and researcher.

Software is changing every domain including Marketing, Healthcare, and even Agriculture. Government is one specific area where the potential for improvement is obvious. Governments are often defined by abstract rules, transactions, permissions and hierarchies, grounded on some fundamental principles – this is striking similar to the same tenets that software and computer science are based on. In that sense, could software be leveraged in this unique industry to create massive change? OpenGov is one company trying to find out through application of software in local city and state governments. The company currently offers a web-based financial visualization software that focuses on making government budget and financial data accessible by modeling and presenting it in a user-friendly way. But it is not farfetched to see the potential being greater and broader in scope. In this episode, Jeff discussed with Andrew how OpenGov is using technology to improve government. Their conversation focuses in on the front-end technologies used to create the webapp, as well as “microservice fatigue” and the other various architectural concerns within the team. Andrew Clark is a software engineer at OpenGov.

In this episode, Jeff and Derrick sit down to discuss the state of tech journalism, and how it has kept up with the rapid pace of technology innovation. Additionally, they discuss the evolution of the “big data” stack and where it is headed. The conversation focuses in on Apache Mesos and Mesosphere, the company building the Datacenter Operating System (DCOS). Derrick Harris is a long time tech journalist, having worked at GigaOm and Fortune as a writer covering tech topics. He currently works at Mesosphere, where he is a tech evangelist.

Incarceration in the United States is often a punitive exercise rather than a rehabilitative one. As a consequence, recidivism is the norm – according to a 2012 report by the California Department of Corrections and Rehabilitation, just over 65% of those released from California’s prison system return within three years. San Quentin State Prison in California is challenging this issue by trying to equip inmates with valuable skills while they are incarcerated. The Last Mile is a program that works with prisoners to help them build relevant skills in technology so that they can more easily transition to productive employment once they are out of prison. Specifically, it teaches inmates how to program, oftentimes starting at the basics like how to use a computer, and how hypertext works. In addition to the challenges of teaching students who may not know computer basics, the program has to work with the constraint of not having internet access. In this episode, Wes and Jeff discuss the realities of incarceration in the United States, and how teaching inmates how to code is a remarkably empowering way to combat and weaken the prison-industrial complex. Wes Bailey is the Director of Program Operations at The Last Mile, and a self-taught developer.

Governments are often laggards in the adoption of new technology. Every aspect of life, from transportation to healthcare, are being affected by the convergence of several trends including connected mobile devices, cloud storage and distributed systems. Yet federal services are in many cases still incapable of delivering the best and most affordable services to citizens. The most visible instance of the U.S. government struggling to modernize was the implementation of Healthcare.gov. What was originally a modest budget for deploying the web marketplace of the Affordable Care Act ballooned into a costly implementation that required hiring external consultants and developers to fix. In response to this dysfunction, 18F was formed with the mission to simplify the government’s digital services. Sarah joins Software Engineering Daily to explain how and why 18F was formed, and how it plans to improve government from within in the years to come. Sarah Allen is a Presidential Innovation Fellow who serves as a developer and product manager for 18F. She is also a co-founder of Bridge Foundry, an organization that helps encourage and empower people who are learning to code.

Riak is a distributed NoSQL key value data store. It was created by Basho, taking inspiration from the Amazon Dynamo paper. In this episode, Jon and Jeff discuss the benefits and different implementations of non-relational distributed data stores, and why Riak is a compelling option. Jon Meredith is the chief architect at Basho.

Apache Spark has skyrocketed in popularity lately, arguably surpassing Hadoop as the hottest big data technology as of late. Even IBM has thrown its weight behind the framework, calling it the “most important new open source project in a decade”. In this episode, Holden joins Software Engineering Daily to discuss why Spark is growing in popularity and how developers can begin learning the framework. Holden Karau is a principal engineer at IBM working with Apache Spark. She is also an author of Learning Spark, a technical guide for developers new to the data processing framework.

Apache Drill is a schema­-free SQL query engine for Hadoop, NoSQL, and cloud storage. Drill is unique in that it uses a JSON data model instead of a relational model like many traditional engines. Additionally, it supports on-the-fly schema discovery, which allows execution to begin without knowing the structure of the data. As a result, Drill can work with data that is constantly updating or changing. Tomer Shiran is the founder of the Apache Drill project and the CEO of Dremio.

Machine learning is something that many business are starting to tack onto their existing processes. Yet, to add machine learning capabilities after the fact is often a fool’s errand. Joshua argues that machine learning cannot be an afterthought, but rather must be custom developed to suit the specific problem or question that each company is trying to answer. His company, Wise.io, tackles this challenge of helping business build ground up machine learning applications that generate accurate predictions for use in an array of business processes. Joshua Bloom is the cofounder and CTO of Wise.io. He is also an astrophysicist, and a professor of astronomy at UC Berkeley.

Data science has emerged as an integral discipline alongside software engineering. In this episode, Jeff and Srini discuss the rise of the field, what it means to be a data scientist, and how the field will evolve as more businesses adopt data driven decision making. Srini also explains his thoughts on the essential qualities of a data scientist, and how educating data scientists is a growing challenge in a world that is flooded with “big data”. Srini Kadamati is a data scientist at Dataquest, where he creates content to help people learn data science.

WebTorrent is a streaming torrent client for the browser that can be used without any additional plugins, extensions, or installations. It is written entirely in JavaScript, and uses WebRTC to handle peer-to-peer communication. There are some limitations to making BitTorrent work over the web using WebRTC, including the inability for a browser-based WebTorrent client or “web peer” to communicate directly with a torrent client. However, the potential for every browser to contribute to the hosting and streaming of files is unprecedented. Imagine if large content sites like Netflix or YouTube did not have centralized hosting, and instead relied on its users to seed the content to each other using their web clients. This enables not only more efficient networks, but also the ability for each user to contribute to and be part of the content delivery network. Feross Aboukhadijeh is the creator of WebTorrent. Previously he built PeerCDN, a peer-to-peer content delivery network, which was acquired by Yahoo in 2013.

Django is an open source web development framework that was initially released in 2005, around the same time as Ruby on Rails. In many ways, the two frameworks are similar – they expedited web development by modularizing common components of web apps and enabling developers to quickly prototype and launch their products. They both encouraged the use of the MVC architectural pattern, with each respectively acting as the controller, allowing developers to use their own front end and database technologies. But the path of the two frameworks diverged in other parameters, including mainstream adoption. Ruby on Rails became the sweetheart in Silicon Valley and elsewhere, while Django was loved by a core constituency of pythonistas who had used Python in other applications and fallen in love with it. At the peak of the frameworks’ popularity, there was a common question from new developers asking whether they should first learn Django or Rails – the answer often boiled down to whether you wanted to use Python or Ruby. In this episode, Frank discusses the history and growth of Django, and explains how it evolved in comparison with other frameworks. Frank Wiles is the President of the Django Software Foundation, and partner at Revolution Systems. He worked at the Lawrence Journal-World newspaper, where Django was initially conceived and created.

Ruby on Rails is a web app framework released 10 years ago that influenced the way websites were being built. Rails skyrocketed to popularity in the late 2000’s and empowered many small companies to quickly build and maintain robust web apps. While it is still a mainstay in web development, it has been overshadowed as of late by JavaScript, and Node.js on the backend. David joins Software Engineering Daily to discuss how Ruby on Rails has evolved to keep up with newer programming paradigms and why it is still an excellent choice to build web apps with nowadays. David Heinemeier Hansson is the creator of Ruby on Rails and a partner at Basecamp. Rails was extracted from the work done at Basecamp, and open-sourced for the public to commit to and use for their applications.

Marty Haught is a director of Ruby Central, a nonprofit dedicated to the support and advocacy of the worldwide ruby community. Ruby Central also hosts the RubyConf and RailsConf software conferences.

Michael Hartl is the author of The Ruby on Rails Tutorial, a widely acclaimed guide to learning how to build Ruby on Rails webapps.

Soylent is a meal replacement beverage, available in both liquid and powdered forms. Soylent is designed with the goal of meeting all nutritional requirements for an average adult. John Coogan is the CTO of Soylent.

Druid is a column-oriented distributed database that excels as a data warehousing solution for fast queries on large data sets. Fangjin Yang is a core contributor to Druid, and is currently co-founder and CEO of Imply, which helps build interactive analytics powered by Druid.

Brian Kernighan is a professor of computer science at Princeton University and the author of several books, including “The Go Programming Language” and “The C Programming Language”, a book more commonly referred to as K&R. Professor Kernighan also worked at Bell Labs alongside Unix creators Ken Thompson and Dennis Ritchie and contributed to the development of Unix.

Kitematic is an open source project built to simplify and streamline using Docker on a Mac or Windows machine. It allows users to get up and running on Docker with a single click, and provides a user interface that makes running and managing Docker files easy. Kitematic was acquired by Docker in March 2015. Sean Li is the co-founder and Chief Product Officer of Kitematic. He is now working on product design and developer experience at Docker.

Distributed computing is becoming an increasingly important and complex field within software engineering, with numbers of new data engineering and infrastructure frameworks being released each year. For a beginner, it can be tough to know where to start, so this episode may be considered a primer to learning about distributed systems independently. Alvaro Videla is a core developer for RabbitMQ and a distributed systems blogger, and he joins Software Engineering Daily to explain some of the core concepts of distributed systems.

Michael Hausenblas is a developer and cloud advocate with Mesosphere, which builds the Datacenter Operating System (DCOS), a distributed OS that uses Apache Mesos as its kernel. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Vue.js is a lightweight front-end JavaScript framework that makes it simple to begin prototyping and building web interfaces. Vue.js provides a flexible API for MVVM data bindings, and can serve as an alternative to other front-end frameworks like Angular and React. Evan You is the creator of Vue and a core developer at Meteor. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Quora is a Q&A website where questions are asked, answered, edited and organized by its community of users. Shreyes Seshasai is the Director of Engineering at Quora. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Triplebyte is a technical hiring platform that vets engineers using a comprehensive evaluation platform and connects them to companies that are interesting in hiring them. Triplebyte was part of the Y Combinator summer class of 2015. Ammon Bartram is the Chief Data Officer and cofounder of Triplebyte. Previously, Ammon was lead video developer at Justin.tv and also cofounded SocialCam. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Eric Elliott is a JavaScript advocate, and the author of Programming JavaScript Applications. He is also an entrepreneur tackling social issues including homelessness and inequality in the tech industry. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Tessel is an open-source microcontroller that is programmable in JavaScript and compatible with Node.js. Kelsey Breseman is a hardware engineer and Steering Committee Member on the Tessel Project. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

The stream processing paradigm is increasingly being adopted by applications that need to process and handle large volumes of data. Apache Kafka is an open-source distributed publish-subscribe messaging system that is built to support streaming data processing. Neha Narkhede is the one of the creators of Apache Kafka, which she built to address engineering challenges while working at LinkedIn. She is also the co-founder of Confluent, which builds enterprise products around Kafka. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Code Fellows in-person code school offering web and mobile development training in Seattle, Portland, and Chicago. Dave Parker is the CEO of Code Fellows and organizer of Seattle Startup Week. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Scott Hanselman is a technologist, teacher and podcaster who works for Microsoft on its web platform team. Scott hosts several podcasts including Hanselminutes and This Developer’s Life, which cover various topics in software. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

TensorFlow is an open source machine learning library intended to bring large-scale, distributed machine learning and deep learning to everyone. Google recently released the framework to the public as a second-generation API, having learned from the successes and failures of DistBelief. Greg Corrado is a senior research scientist and tech lead at Google, where he focuses on the research areas of machine intelligence, machine perception and natural language processing. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Rebecca Garcia is the creator of GeekGirlWeb and founder of CoderDojo NYC. She spends much of her time educating people about the importance of STEM education, particularly in the domain of programming. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Todd Bishop from The GeekWire Podcast recently interviewed Jeff for a discussion on various topics including programming languages, coding bootcamps and the philosophical question of whether programming is an art or a science. Thanks to GeekWire for the opportunity! You can find the show and more at Geekwire: http://www.geekwire.com/2015/is-software-development-art-or-science-geekwire-podcast-with-software-engineering-daily/

Spotify is a streaming music service that uses data science and machine learning to implement product features such as recommendation systems and music categorization, but also to answer internal questions. Boxun Zhang is a data scientist at Spotify where he focuses on understanding user behavior within the product. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

The Internet of Things era is upon us, and with it the related concerns of security, privacy and reliability of the connected systems. This episode explores these issues and how companies can prevent these problems through better development processes and lifecycle management. Anders Wallgren is the CTO of Electric Cloud, which builds products to help companies optimize their software build, test, and deployment process. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

John K. Paul is an engineering manager and speaker, who has given several talks on Javascript, including JavaScript, the Real Bad Parts. He is the CTO of InRhythm and also an organizer of NYCHTML5. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Richard Golden is the host of Learning Machines 101, a podcast that covers artificial intelligence and machine learning topics. Dr. Golden is also a full-time Professor of Cognitive Science and Electrical Engineering at UT Dallas. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Real-time stream processing of data is becoming widely adopted in the efforts to manage and process “big data”. Some of the top frameworks for processing data streams include Storm, Spark, Samza and Flink. Satish Mittal is an architect at InMobi, an ad platform that needs to deal with processing large volumes of ad data with tight time demands. He recently conducted an analysis of streaming frameworks to determine which of the tools was right for the application at InMobi. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Uber is a transportation and logistics company that manages many aspects of its ride-sharing services through mobile apps and distributed technology. Uber faces unique challenges in rapidly scaling its services internationally, and at one point increased its developer headcount from 200 to over 1000 in less than a year. Matt Ranney is the Chief Systems Architect at Uber and was previously a founder and CTO of Voxer. At QCon San Francisco, he gave a talk called Scaling Uber. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

React Native is leading to a future where a developer can build native experiences on web, iOS and Android platforms without having to write 3x the code. Christopher Chedeau is an engineer at Facebook who works on the React team. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.

Data engineering is a newly emerging field that exists at the intersection of software engineering and data science. Data engineers are increasingly in demand for their abilities to use the various open source big data frameworks to build efficient and robust data pipelines. David Drummond and Austin Ouyang are program directors for the Insight Data Engineering Fellows Program, where they help train data engineers that can immediately start contributing in the industry. Hired.com is the job marketplace for software engineers. Go to hired.com/softwareengineeringdaily to get a $600 bonus upon landing a job through Hired. Digital Ocean is the simplest cloud hosting provider. Use promo code SEDAILY for $10 in free credit.