Software Engineering Daily

Many programmers listening to this podcast are working at a big company, and they would prefer to be running their own software business. Patrick McKenzie has been writing about this topic for several years on his blog Kalzumeus.com. Almost a decade ago, he was working as an enterprise developer at a large company in Japan. Over time, his side projects started making enough money to justify his leaving to work on them full-time. Starting a small successful software business is not about luck, or taking big risks, or having your muse deliver some brilliant insight. It is about studying the markets where you can build software that can deliver value. Patrick and I discuss this topic, as well as Stripe–where he now works today. He sold his small businesses to go work full time on Stripe Atlas, so his story of transitioning back to a big company is fascinating as well.

Botnets have a massive influence on the Internet. As we have seen recently with the Mirai Botnet, IOT bots can take down companies as big as Netflix. In our recent episodes about advertising fraud, we’ve talked about how bots are being used to take billions of dollars of revenue from advertisers. Derek Muller is one of those advertisers who has spent money on ads and gotten nothing but fake traffic in return. Two years ago, he posted a video on YouTube about his experience purchasing advertising traffic on Facebook, and getting Likes from accounts that were clearly from fake accounts. Derek is the host of Veritasium, an awesome YouTube channel about science, truth, and technology–so we also talked some about how he built a successful YouTube business.

Slack is a chat client that has reached wide adoption. The rise of Slack has coincided with the rise of chatbots. A chatbot is a simple, conversational interface into a computer program that may have simple functionality, like telling you some simple statistics, or more complex functionality, like helping you manage your continuous integration pipeline. Bot design and engineering is a new field, and a vast array of resources and techniques are available for developers looking to hack on it. Amir Shevat is the director of developer relations at Slack, and is responsible for communicating with developers about the best ways to build bots for Slack. After seeing his talk at O’Reilly Bot Day, I had a number of questions about where we are with bots today and where we are going. I enjoyed this conversation with Amir, and if you aren’t already convinced that bots are an important platform for engineers to understand, this conversation will convince you.

Large technology companies have no shortage of data. But raw data itself does not provide a competitive advantage. Many companies are bottlenecked by a shortage of data scientists who can query that data effectively. This results in an organizational dysfunction where people lining up to ask questions of the data science team are unable to move as fast as they want to. Tomasz Tunguz saw this when he was working at Google, and he continues to see it today from his position at Redpoint, where he works as a venture capitalist looking at companies. The problems with the “data supply chain” led him to write “Winning With Data”, a book about how technology companies can successfully operationalize, explore, and act on their data. We’ve done many shows about the interactions between software engineers, data engineers, and data scientists. This episode is a great complement to previous episodes, providing a holistic view into the way data moves through an organization–and I highly recommend Tomasz’s book “Winning with Data”.

Amazon Web Services changed the economics of building an internet application. Instead of having to invest tens of thousands of dollars up front for hardware, developers can pay for services over time as their application scales. As AWS has grown to be a gigantic platform, the documentation about how to use cloud infrastructure has become insufficient. As an answer to this, Joshua Levy initiated The Open Guide to Amazon Web Services, an open-source collection of resources available on github. Joshua has experience at a variety of companies, including Viv, the conversational interface that was recently acquired by Samsung. In our conversation, Josh brought his years of experience to the table to explain the risks, benefits, and alternatives of Amazon Web Services. Josh has become a friend over the last year, and if you get the chance to have a conversation with him, I highly recommend it. He will be at the re:Invent conference in Vegas at the end of this month (November) and if you are interested in talking to him about the AWS Open Guide, or anything else, reach out to him.

Bot Day was an O’Reilly conference featuring talks from some of the leading figures in the bot industry. Before I attended Bot Day, I knew there were lots of applications for chatbots, but I didn’t realize how good the tooling has gotten–it’s very easy to get started with chatbots today so if you are a developer and you have a basic idea, I suggest checking out the videos from Bot Day, as they are a great resource for how to get started. When I was at the conference, I had a number of conversations with Edaena Salinas, who hosts The Women In Tech Show. We also caught up a few weeks after the conference to discuss our reflections on Bot Day.

The blueprint for a typical startup involves investing heavily in cloud services–either from Amazon, Google, or Microsoft. The high costs can quickly eat away at all of the money that startup has raised. In today’s episode, Avi Freedman outlines some of the infrastructure mistakes that can set back a company severely–cloud jail, hipster tools, and lack of monitorability. Avi is the CEO of Kentik, a network traffic, performance, and security visibility company, so he gets lots of perspective on the infrastructure of the average company. We also discussed the business of running a software company, contrasted with the life of a poker player. Avi and I both have some experience playing high stakes poker, so it was a great opportunity to get his perspective on the parallels between the two fields.

When Facebook acquired Instagram, one of the first systems Instagram plugged into was Facebook’s internal spam and fraud prevention system. Pete Hunt was the first Facebook engineer to join the Instagram team. When he joined, the big problems at Instagram were around fake accounts, harassment, and large volumes of spammy comments. After seeing the internal Facebook spam prevention tools clean up Instagram, Pete decided to start a company to build products that would allow this type of spam and fraud prevention as a service. Smyte provides protection against bad actors on the Internet. Complex marketplaces like Tilt, Meetup, and TaskRabbit plug into Smyte to filter their transactions for suspicious behavior. In this episode, we talk about how this type of behavior manifests, and the event-driven software architecture that the team at Smyte has built.

Ben Northrop was sitting at a tech conference, listening to a presentation about a new piece of technology, when he was struck by the sense that history was repeating itself. For the twenty years that Ben has worked as a software engineer, he has been hearing about new technologies that claim they will be able to disrupt everything, and he has relentlessly been keeping up as best he can with the latest trends. Of course, Ben is all of us. Nobody can keep up. As a daily software podcaster, I think I am as well positioned as anyone to be tuned into the breadth of the software engineering ecosystem–but I am constantly behind as well. In his blog post “Reflections of an Old Programmer”, Ben theorized about the causes of this feeling of fatigue, and the constant struggle to keep up with the world of software engineering. His post contains enduring lessons that will be of value to new programmers and will feel familiar to programmers who have been around for awhile.

Jeff Bezanson’s university thesis described the motivation for a new programming language. He discussed the shortcomings of “array based programming environments” and his desire to create a more performant language with the best qualities of Lisp, Python, Ruby, Perl, Mathematica, R, and C. The Julia Language is a high performance language designed to suit technical users that crave the flexibility to pick their own notation. The language has support for a wide variety of operators, allowing scientists to create domain specific equations that can be elegantly expressed as code. Jeff Bezanson joins the show to discuss his motivations for Julia, and how the language has evolved since he started working on it.

Legacy code is code without automated tests. Most companies have lots of legacy code, and most developers don’t like working on legacy code. Why is that? What is it that makes legacy code so difficult to work with? And why does a large amount of legacy code slow down an organization so severely? Andrea Goulet is the CEO of Corgibytes, a consultancy that specializes in remodeling legacy codebases. Andrea’s company developed out of an observation that there are developers who actually love working on legacy code. Since founding Corgibytes, she has become an expert on legacy code and published a recent article about Building Technical Wealth. The article presents ways that companies can confront the realities of legacy code.

Indie Hackers is a website that profiles independent developers who have made profitable software projects, usually without raising any money. These projects make anywhere from a few hundred dollars a month to more than $100,000 as in the case with park.io, one of the services profiled by Indie Hackers. Courtland Allen is the creator, engineer, and interviewer behind Indie Hackers. For each business that is profiled by Indie Hackers, Courtland conducts a short interview with the founder. Courtland joins the show to discuss the changing trends that are making it easier to bootstrap a software business if you are a capable developer–or even if you are a nontechnical person who understands how software works. Since Courtland and I are both in the business of interviewing engineers, we had a lot to talk about, and this is a fantastic episode.

When you visit a web page, that web page can write data to a file on your computer, known as a cookie. Scripts on that page can also read from your cookie file to understand where you have been in the past. All of this data about you is getting shared between advertising companies like Google, Facebook, and AppNexus. Ghostery is a browser extension that allows you to limit what you share with these online tracking companies. Larry Furr develops products at Ghostery, and on this episode he takes us through the process of how we are tracked through the Internet. We also explore the topic of ad fraud, which is a theme we will continue to explore on SE Daily. If you have any information on ad fraud, or have recommendations for guests, please email me at [email protected]

Chat bots are your newest co-worker. Slack, HipChat, and other chat clients allow developers and other team members to communicate more dynamically than the limits of email. Companies have started to add bots to their chat rooms. These bots can give you technical information, restart a server, or notify you that a build has finished. Jason Hand is the author of ChatOps: Managing Infrastructure in Group Chat. He joins the show today to discuss how ChatOps improves development and operations by centralizing lots of functionality in group chat. Edaena Salinas is the host for today’s show. She also hosts the excellent Women In Tech Show–a podcast we highly recommend. Since we are on the subject of bots–we want to thank O’Reilly Media for recently providing Software Engineering Daily a ticket to Bot Day.

Online advertising is heavily affected by a set of delusions and fraudulent practices that few people in the adtech industry have an interest in stopping. This is the curious, perverse nature of the world of online media. Some of the online advertising we see fulfills its job, when an ad successfully conveys a meaningful message from a marketer to a consumer. But there are reasons to be extremely skeptical of the way that online advertising works. In this episode, and several more to come, Software Engineering Daily will be diving deep into the adtech business to understand the bot fraud and mass psychosis that bolsters our tech media giants. Bob Hoffman is the author of The Ad Contrarian, a long-running blog about advertising. After selling the advertising agency he was running, he left the industry. In today’s episode, Bob discusses his experience in the ad industry, and the delusions he saw when he was working in it.

Music collaboration software that works over the Internet is a software challenge that has not been fully tackled. On today’s Internet, users collaborate intensively on programming projects, journalism, and other projects, but the tools for collaborating on music online have not yet become popular. Blend.io is a social music collaboration tool–a github for musicians. I have been using it myself and enjoying it tremendously. Today’s guest Alan Grow joins the show to talk about how to build a social version control system for musicians, as well as where music is headed.

Advertising fraud takes billions of dollars out of the economy every year. We don’t know exactly how much money is being lost, because we don’t know what percentage of Internet users are bots. Are You A Human is a company designed to solve that exact problem, and provide a service for verifying whether a user is real or automated. Ben Trenda is the CEO of Are You A Human. In this episode, we discuss some of the ways that bots pose as humans–clicking on ads, pretending to read ads, and defrauding marketers and publishers. We also talk about how Are You A Human combats this. This episode is part of a series of episodes about advertising fraud. If you know anything about the area, send me an email: [email protected]

Blockchain technologies like Bitcoin and Ethereum have not impacted the lives of most consumers today. The theoretical breakthroughs that blockchain enables will eventually happen–I will be able to pay 1 cent to a knowledge worker in Africa without having to pay a 5 cent transaction fee. My servers will be able to pay other servers for small compute jobs. We will have decentralized versions of sharing economy systems like Airbnb and Uber. But first there are many infrastructure and policy issues to sort through. The technology is not quite there yet, and people running even simple mining operations are threatened by our current legal framework. Coin Center is an organization dedicated to education of policymakers and media about blockchain technology. Peter Van Valkenburgh directs research at Coin Center, and joins the show to talk about the issues that stand in the way of that utopian dream of micropayments, IoT, and other blockchain miracles.

Kafka is a distributed log for producers and consumers to publish messages to each other. We’ve done many shows about Kafka as a key building block for distributed systems, but we often leave out the discussion of the complexities of setting up Kafka and monitoring it. Kafka deployments can be a complex piece of software to manage. Tom Crayford is an engineer at Heroku, where he helped engineer the recent Heroku Kafka product, which is a managed version of Apache Kafka. Our conversation explored the use cases of Kafka and how to build Kafka as a cloud service at scale. For more information about Heroku Kafka, check out an upcoming webinar. Full disclosure: Heroku is a sponsor of Software Engineering Daily. That said, this is a topic I am genuinely interested in–it is often difficult to get cloud providers to talk in detail about how they are architecting their services.

Netflix has a highly interactive user interface. As I move my mouse around the page, hovering over titles and inspecting movie descriptions, there is a lot going on under the hood. One component of this UI is RxJS, a library for building reactive JavaScript. Reactive programming uses the observer pattern to create objects that emit streams of events. We can compose these streams together to create elegant abstractions. Reactive programming may seem confusing at first, but it can simplify certain patterns that may be hard to describe with imperative programming. Ben Lesh, a senior software engineer at Netflix, joins the show to explain why reactive programming is useful, and how RxJS is used at Netflix.

Bot fraud, the New York tech scene, RethinkDB and open source; these topics and more are discussed in today’s episode. Two of the most popular guests return to the show to explore a variety of topics. Ben Halpern is the creator of The Practical Dev, a massively popular Twitter account and blog that you may recognize from its parody O’Reilly book covers. Haseeb Qureshi is an engineer at Airbnb, and a blogger who is well known for his writings on salary negotiation and coding boot camps. This episode was an experiment–if you like it, please let me know what you think.

Google Compute Engine is the public cloud built by Google. It provides infrastructure- and platform-as-a-service capabilities that rival Amazon Web Services. Today’s guest Joe Beda was there from the beginning of GCE, and he was also one of the early engineers on the Kubernetes project. Google’s internal systems have made it easy for employees to spin up compute resources, but it was not a simple task to make this internal cloud consumable by the public–not to mention competitive with AWS. In order for a cloud provider to be successful, it needs to offer self-healing, self-managing infrastructure that can run microservices.

Cloud computing was something much different in 2011, when Brian Gracely and Aaron Delp started The Cloudcast, a podcast I listen to on a regular basis. The Cloudcast features technical discussions about cloud infrastructure technology, and one of the most recent shows was a monologue by Brian Gracely where he explained his perspective on the industry rumblings about a Docker fork. The utility of a container for so many different purposes leads to different organizations having differing preferences for what use case is optimized for. The impression that I took away from this conversation, as well as the next episode that will air with Joe Beda, is that the diverse opinions and products in the container and orchestration ecosystem is quite healthy. Brian does a great job explaining his perspective on The Cloudcast, and he discusses his beliefs further in this episode.

The Simpsons is a classic, beloved television show. The scripts of The Simpsons have been made publicly available, and include dialogue, location, and character information. Todd Schneider used these scripts and other information sources as a corpus to analyze The Simpsons and find interesting statistics–such as who the most important supporting characters were, and how the ratings of the show have trended over time relative to other TV shows that have declined in ratings. Todd works at Genius in New York, and I took the opportunity to ask him other questions about pop culture, given that Genius is a place where pop culture and data collide. If you haven’t seen the site, it started out as a wiki for rap music, and quickly grew to more aspects of pop culture. Todd also has a variety of other side projects, involving betting markets, analysis of taxis vs Lyft and Uber, and a systematic study of the wedding section of the New York Times.

Data engineering is the software engineering that enables data scientists to work effectively. In today’s episode, we explore the different sides of data engineering–the data science algorithms that need to be processed and the implementation of software architectures that enable those algorithms to run smoothly. The PANCAKE STACK is a 12-letter acronym that Chris Fregly gave to a collection of data engineering technologies including Presto, Cassandra, Kafka, Elastic Search, and Spark. In his current life, Chris travels around the world giving workshops on how to deploy and use the PANCAKE STACK. Before that, he was an engineer at Netflix, where he received an Emmy for Streaming Engineering Excellence.

When a user of a social network updates her profile, that profile update needs to propagate to several databases that want to know about such an update–search indexes, user databases, caches, and other services. When Neha Narkhede was at LinkedIn, she helped develop Kafka, which was deployed at LinkedIn to help solve this very problem. Using Kafka as an event queue, LinkedIn adopted the CQRS architectural pattern together with event sourcing. Event sourcing is an architectural pattern that allows changes to our application model to be represented as events. Each event is published to an event queue, and is pulled off of the queue by each of the various services that need to consume that event. Event sourcing and the related architectural pattern CQRS allow for a flow of information through an application that is easy to reason about, and has several other desirable properties. In today’s episode, Neha explains how to use Kafka for event sourcing and how related software patterns are improving the architectures of companies like Netflix and Uber. For more information, check out this Confluent blog entry.

The intent of the DevOps movement is to get organizations moving faster and more effectively by breaking down siloes, and improving communication. Gene Kim’s book The Phoenix Project illustrated this by telling the fictional story of a company adopting a DevOps mentality. Although that book was fiction, Gene is an experienced engineer, having worked as founder and CTO of Tripwire, a software company that makes security and compliance automation software. In his new book The DevOps Handbook, Gene presents a practical companion to The Phoenix Project. Together with his co-authors, Gene has written a guide for how to move an organization toward DevOps, and in this episode we explore some of the topics from his book. Check out an excerpt from Gene’s book here: DevOps Handbook

At Netflix, developers write applications with a variety of requirements–from simple requests for a list of movies to more resource-intensive requests like a complex machine learning workflow. Netflix wants developers to be able to request the resources they need from a compute cluster and receive those resources on-demand, without thinking too much about the state of that pool of resources they are drawing from. At the cluster level, this means scheduling the application requests intelligently. Sharma Podila, a distributed systems software architect at Netflix, explains how Netflix has built a cloud native scheduling system on top of Mesos.

Building a monitoring system is a complex distributed systems problem. Events are produced from different points in an application and must be aggregated in order to form metrics. These events are often ingested by a time series database, which forms the backbone of our monitoring system. Theo Schlossnagle is the CEO of Circonus, where he has been working on architecting the company’s monitoring software for six years. In this episode, we talk about how to build a monitoring system and the requirements for the underlying time-series database, as well as what monitoring even is.

In order to move software updates from the development team to production, companies do a variety of things. Some teams might email files to each other or use FTP or even floppy disks. Most companies today at least use version control systems like Git together with separate servers for development and production. When code is ready to move to production, a build that is on the development server gets copied over to the production servers, and the production servers begin serving real users. This process is known as deployment, and over the last few decades companies have started deploying more rapidly (even “continuously”), leading to faster iterations and better feedback loops between the software development team and the users of the product. A particularly effective version of this workflow is known as continuous delivery. In today’s episode, David Rice from ThoughtWorks joins the show to give a short history of continuous delivery, and how continuous delivery actually looks in practice.

Kafka Streams is a library for building streaming applications that transform input Kafka topics into output Kafka topics. In a time when there are numerous streaming frameworks already out there, why do we need yet another? To quote today’s guest Jay Kreps “the gap we see Kafka Streams filling is less the analytics-focused domain these frameworks focus on and more building core applications and microservices that process data streams.” Jay is the CEO of Confluent, a company that is building Kafka technology, and he is one of the original authors of Kafka. We began with a brief history of why Kafka evolved to be the message broker of choice for so many data engineering stacks, then we got into a discussion of streaming–what is streaming? How does Kafka communicate with streaming frameworks? Our conversation concluded with a discussion of Kafka Streams and the future of the Kafka data platform.

Scala is a functional programming language built on the JVM. For more than a decade, this didn’t mean anything to front end web developers. More recently, ScalaJS has brought Scala to the front end. ScalaJS is a project that compiles any Scala program down to JavaScript–so that all of your Scala programs can run on the browser. Haoyi Li has worked on ScalaJS extensively and has written an online book about ScalaJS. If you are a front end developer looking for a safer way to write your web apps, you will like this episode–or if you are a fan of functional programming.

Software teams are traditionally composed of roles such as project manager, developer, QA, and manager. What happens if you throw out all of those titles, hire mostly engineers, and ask them to do whatever they think is best? That is the core idea behind Fred George’s idea of Developer Anarchy. In today’s episode, David Curry guest hosts an interview with Fred George. They talk about the structure of teams and the idea of developer anarchy. For a great complement to this episode, check out the Software Engineering Radio episode about developer anarchy.

Platform as a service can mean different things to different people. The most prominent feature of a PaaS is the ability to abstract away issues that every developer within an organization has to deal with. As an example, developers today don’t need to fear scalability and load balancing issues as much as engineers of the 90’s and early 2000s. We can develop our applications without worrying about the scaling that is going on under the covers. Sinclair Schuller is the CEO of Apprenda and he joins the show today to discuss what is involved in building a PaaS, particularly one targeted at enterprises. His company was started in 2007, just a few years after AWS got started. We also talked about the potential Docker Fork, for which Sinclair had useful business and engineering perspectives.

Docker containers changed how engineers think about building software, and the company most responsible for the widespread adoption of containers is Docker itself. Since containerization has caught on in the mainstream, companies like RedHat, Google, Huawei, and many other big players have built platform products that utilize Docker containers. Docker containers are the unit that many engineers use to deploy their applications, but the servers that run those containers are usually on infrastructure providers like Amazon, Google, or Rackspace. This raises the question–how will Docker the company make money? Having raised $180M in equity, Docker has plenty of time to find something that works, but there are suspicions in the open source community that aggressive changes to the open source Docker project are being made by Docker the company with the long-term objective of monetization. These changes–namely the default support for Docker Swarm–are not nefarious, but they might be less likely to occur of Docker were just an open source project, as opposed to an open source project being tended by a well-financed company. The New Stack broke the story about the potential Docker Fork, and Alex Williams and Joab Jackson of The New Stack join me on today’s episode. We go deep into the facts and speculations about why a fork might occur, the motivations of different key players, and the disputes within the community that have led us to this moment.

Africa has a huge reservoir of untapped technical talent. Tech companies have a huge amount of work they need skilled employees for. Andela recruits the most talented technologists across Africa, shapes them into world-class software developers, and places them with tech companies worldwide such as Google and Microsoft. Brice Nkengsa is the director of engineering at Andela, and he joins the show to discuss Andela. This episode touches on many of the themes of Software Engineering Daily–boot camps, scaling an organization, microservices, and the future of work.

After nine years at Microsoft, Scott Berkun left to become an author. One of his books on project management was read by Matt Mullenweg, the creator of the WordPress blogging tool that runs a large percentage of the internet (including Software Engineering Daily). Scott became friends with the WordPress founder, who is also the CEO of Automattic, a company that sells WordPress hosting and support. Around that time, Automattic was run as a completely flat company. Everyone reported to Matt. This was unsustainable, and eventually Matt decided that Automattic should try management, and he convinced Scott to take a break from his writing career to return to the world of software engineering as Automattic’s first manager. Scott accepted the role under the condition that he could write a book about the experience, and the result of that agreement is “The Year Without Pants: WordPress.com and the Future of Work”, which is a brilliant depiction of what it is like to work at a completely remote software company. Scott joins the show today to talk about his book and his career journey–from Microsoft, to writing, to WordPress, and back to writing. He also shares his thoughts on the workplace dynamics that we spend our days in at software companies.

Mark Zuckerberg may be the most powerful person in the world. At no other time in history has a single human had such fine-grained control over the most influential tool for media. Today’s guests are Michael Zimmer and Nick Proferes, the creators of The Zuckerberg Files, an index of every recorded word that Mark Zuckerberg has said in text, video, or audio. Why would someone create this? I was partly expecting Michael and Nick to be anti-Facebook, because The Zuckerberg Files sounds like a tool to hold the Facebook CEO accountable. In reality, Michael and Nick have a nuanced view of Facebook and Zuckerberg–they are Facebook users themselves, and they initially built The Zuckerberg Files as a way to portent product directions that Facebook might lean towards–particularly those related to privacy.

Scikit-learn is a set of machine learning tools in Python that provides easy-to-use interfaces for building predictive models. In a previous episode with Per Harald Borgen about Machine Learning For Sales, he illustrated how easy it is to get up and running and productive with scikit-learn, even if you are not a machine learning expert. Srini Kadamati hosts today’s show and interviews Andreas Mueller, a core committer to scikit-learn. Srini and Andreas discuss the background and implementation of scikit-learn and walk through some prototypical workflows for using it.

Containers have become the unit of infrastructure that many technology stacks deploy to. With the shift to containers, the attack surface of an application has changed, and we need to reconsider our security models; the resource allocation of our containers, the interactions between different containers on a single machine, and the big picture–how the external web may interact with our containers. Phil Estes joins the show to discuss container security, as well as the OCI, container orchestration, as well as other container related topics.

Security for the popular chat application Slack is a major focus for the company. A corporate Slack account is as valuable to a hacker as a corporate email account. In today’s episode, Ryan Huber and I talk through Slack’s approach to security–from philosophical discussions of how to company approaches security to the technical practices of logging and monitoring, and why Slack has a separate Amazon Web Services account just for the security team. Ryan works on security at Slack and has written a Medium post called Distributed Security Alerting, that I recommend checking out. If you haven’t heard it, you might also like the previous episode we did about Slack’s Architecture, with Keith Adams from Slack.

The role of “tech lead” is a combination of a software engineer, a project manager, and an architect. A tech lead might spend 30% of her time coding and 70% managing a project, resolving conflicts, and planning. In today’s episode, we discuss the structure of software teams, and when it might make sense to have a tech lead on your team, in contrast to the more common team structure of project managers, engineers, and engineering managers. Jeff Norris, an engineer from ThoughtWorks joins the show to discuss the idea of the tech lead. This is a great show for anyone in management who is looking for alternative team structures, and this episode might also appeal to engineers who are looking to transition to a role with more variety.

Devoxx4Kids is an organization that is inspiring children through robotics, programming, and engineering. Kids who attend a Devoxx4Kids workshop are exposed to entertaining and educational activities, such as writing a Minecraft mod or programming an Arduino. Arun Gupta is a software engineer and the president of the board for Devoxx4Kids USA. In this episode we discuss coding education for kids, and what happens at a Devoxx4Kids workshop. If you have a kid who you want to expose to technology, this episode has some great suggestions for how to do that.

Google builds cloud services for developers, such as PubSub, Cloud Storage, BigQuery, and Cloud DataStore. On Software Engineering Daily, we’ve done lots of shows about how these types of services are built. In this episode, we are zooming in on the interaction between the developer using a cloud service and the design and engineering of the client APIs. To build a useful cloud service, Google has to make a common way of interacting with that service from any programming language on any device. Jon Skeet is a longtime engineer at Google, and he joins the show to explain how Google uses generated code to make the creation of those APIs more streamlined. We talk about gRPC, protocol buffers, C#, and lots of other topics.

When the US government hacks its own citizens, The Electronic Frontier Foundation is often the best source of reporting to find out what laws the government has broken. When a change to the privacy policy of Google or Facebook is made, the Electronic Frontier Foundation is the best place to find out how that change in privacy exploits users. The Electronic Frontier Foundation provides legal defense and editorialism at the intersection of law and technology. Nate Cardozo joins the show today to discuss the mission of the EFF and his work at the organization. We have a wide ranging conversation, ranging from governments to corporations to Stuxnet to how the internal discussions at the EFF lead to the stances taken by the EFF.

Batch and stream processing systems have been evolving for the past decade. From MapReduce to Apache Storm to Dataflow, the best practices for large volume data processing have become more sophisticated as the industry and open source communities have iterated on them. Dataflow and Apache Beam are projects that present a unified batch and stream processing system. A previous episode with Frances Perry discussed how they work in detail. In today’s episode, Eric Anderson discusses Cloud Dataflow, a service from Google that lets users manage their data processing pipelines without having to spin up individual servers to run them on. Cloud Dataflow–like the “serverless” movement we have done several shows on–represents a growing shift towards cloud providers offering services that abstract away the operational challenges of managing compute nodes. If you have suggestions for topics in this area, please do send me an email.

A previous episode of Software Engineering Daily called “You Are Not A Commodity” received a lot of feedback, both negative and positive. The episode was a monologue I wrote about why engineers should build products on their own as a default career path, rather than work at a large corporation as a default career path. A reddit thread about the episode was almost entirely negative. Most of the emails I received about the episode were very positive. Other regular listeners wrote in and said they had mixed feelings. One of those listeners was Preethi Kasireddy, a previous guest of the show. In this episode, Preethi gives some counterarguments and questions the ideas that were presented in the “You Are Not A Commodity” episode. This will be a useful discussion for anyone who is thinking about their career path, and whether to go work for a company or do something on their own.

Unikernels allow us to specify the minimum features of an operating system we need to deploy our applications. We’ve had many shows about containers, which allow you to deploy your application on top of a segregated portion of an operating system. Unikernels are different because they can be deployed directly to bare metal or to a hypervisor. Idit Levine works on Unikernels at EMC. We had an interview a few weeks ago with her colleague Scott Weiss, and today’s conversation goes deeper into the motivations for Unikernels, the problems they solve, and the project she is leading called Unik, which seeks to add the degree of usability and adoption to Unikernels that Docker brought to containers.

Microsoft was the dominant technology company in the 1990’s, until it came under fire for anticompetitive practices. Internet Explorer was tightly coupled to the Windows operating system, which prevented Netscape Navigator–a competing browser–from reaching users on the dominant platform. This episode is about antitrust–what businesses can and cannot do in the name of competition, what the impact of Microsoft’s legal battles in the late 90’s was, and how the law might respond to potential technology monopolies in the near future–such as Facebook, Google, and Uber. Our guest Harry First is a professor of law at New York University School of Law. He wrote a textbook called “The Microsoft Antitrust Cases: Competition Policy for the Twenty-First Century”.Sponsors

Slack is a chat application that is rapidly growing in popularity. The focus of Slack is to create a polished, responsive tool for productivity that cuts down on the emailing, context switching, and useless meetings that take place at a typical enterprise. Keith Adams, the chief architect at Slack, joins the show to explain how those high level principles translate into engineering decisions. Keith previously worked for 7 years at Facebook, contributing heavily to the tools that make PHP easier to develop with at scale. Slack’s core product is also built with PHP, and Keith discusses the similarities and differences between scaling Facebook and Slack.