Software Engineering Daily

Designers and software engineers need to communicate with each other. From Apple to Slack to Uber, the emphasis on visual design within a product is rising in importance. Much like development and operations siloes have been bridged with the DevOps movement, design and engineering teams are working more closely together to align the vision of the designers with the realities of code. InVision is a product for prototyping designs and product workflows. I categorize InVision with high-level productivity tools like Trello, Asana, and github. When teams start using InVision, it often becomes an integral part of the entire product workflow. A designer will mock up their product vision on InVision and share it with the rest of the team for criticism and commentary. Since the product is the perfect bridge between engineers and designers, InVision the company is expanding as a platform. Bjorn Freeman-Benson is the CTO at InVision and his task is to scale the 100% remote engineering team while developing new product features. Bjorn describes himself as a software psychologist. We talked about the psychology of managing engineers and the technical challenges of building a large, popular web app for technology products.

Developing a product requires careful balance between engineering, sales, design, and customer service. The founding CTO of a company often needs to take on each of these responsibilities, because when the company only has a few people there is nobody to delegate these different tasks to. Cullen Zandstra is the CTO at FloQast, a SaaS tool for accounting close management. It isn’t important what that means–we do describe it in this episode, but the actual software product that we are discussing is less important than the general discussions we have. Cullen gives detailed explanations for how to develop a software product. In the past, Cullen worked at MySpace. After we discussed his current company FloQast, we discussed some of what he learned at MySpace, and why the social network proved to be less durable than Facebook. We also had some great discussion as to why Google+ was ALSO less durable than Facebook. Also–I should mention that FloQast is hiring engineers in their Los Angeles office. And thanks to my friend Patrick Mathieson for connecting Cullen and myself.

Net neutrality is the principle that internet service providers and governments regulating the internet should treat all data on the internet the same. Debates around Net Neutrality can be as contentious as subjects like global warming, or tabs vs. spaces. To a hardcore free market economist, Net Neutrality sounds suspicious. Why would it be good for the government to regulate prices on the data that passes around wires owned by a corporation? The problem is that a large portion of the United States can only be served by wires owned by a single company. In those instances, the company that owns the wires has monopoly pricing power over the data delivered to homes in those geographic regions. Quincy Larson is the founder of Free Code Camp and a frequent author of internet-based articles on Medium. He returns to Software Engineering Daily to make the case for Net Neutrality. I was convinced by some of his arguments and less convinced by others. In any case, I think you will find this episode entertaining and informative. If you are like me, after this episode you will have at least a slightly better understanding of the issues surrounding Net Neutrality. Inside the Invisible War for the Open Internet

In bitcoin, every transaction in the shared ledger has the sender, recipient, and value. This ledger gets appended infinitely and is shared within a peer-to-peer network. Zcash is a cryptocurrency with all the features of bitcoin, plus encrypted transactions. The sender, recipient, and value fields are all encrypted. If bitcoin is HTTP, Zcash is like HTTPS, a secure transport layer. Nathan Wilcox works on Zcash, and in this episode we discuss why an encrypted version of bitcoin is useful, how mining works, and how Zcash the company is structured. Nathan also gives some context for the current state of the bitcoin community, and where Zcash fits in.

Wearables have become more accessible to the public. Snap’s spectacles, Google Glass, FitBit, and Apple Watch suggest a future in which many people will be wearing a smart device. In this episode Asta Roseway, Research Designer at Microsoft Research, gives insights into other categories of wearables like tattoos, scarves, and cosmetics. Asta talked about her work on DuoSkin, a wearable that looks like a beautiful metallic tattoo. We talked about its capabilities and how it was built, and why it is still too early for connected tattoos. We also talked about the intersection between health, wearables, technology, and fashion and how wearables might look in the near future. This episode is guest hosted by Edaena Salinas, who also hosts The Women in Tech Show. Asta Roseway’s Website Asta Roseway Microsoft Research MoodWings LightWear

Multiagent systems involve the interaction of autonomous agents that may be acting independently or in collaboration with each other. Examples of these systems include financial markets, robot soccer matches, and automated warehouses. Today’s guest Peter Stone is a professor of computer science who specializies in multiagent systems and robotics. In this episode, we discuss some of the canonical problems of multiagent systems, which have some overlap with the canonical problems of distributed systems–for example, the problems of coordinating between different agents with varying levels of trust resembles the problem of establishing consistency across servers in a database cluster. Peter has recently contributed to the 100 year study of artificial intelligence, so we also had a chance to discuss the opportunities and roadblocks for AI in the near future. And since Peter teaches computer science at my alma mater, UT Austin, I had to ask him a few questions about the curriculum.

Biology research is complex. The sample size of a biological data set is often too small to make confident judgments about the biological system being studied. During Jason Knight’s PhD research, the RNA sequence data that he was studying was not significant enough to make strong conclusions about the gene regulatory networks he was trying to understand. After working in academia, and then at Human Longevity, Inc Jason came to the conclusion that the best way to work towards biology breakthroughs was to work on the computer systems that enable those breakthroughs. He went to work at Nervana Systems on hardware and software for deep learning. Nervana was subsequently acquired by Intel. In this episode, we discuss how machine learning can be applied to biology today, and how industrial research and development is key to enabling more breakthroughs in the future. The main lesson I took away from this show is that while we have seen phenomenal breakthroughs in certain areas of health–like image recognition applied to diabetic retinopathy or skin cancer–the challenges of reverse engineering our genome to understand how nucleic acids fit together into humans are still out of reach, and improving the hardware used for deep learning will be necessary to tackle these kinds of informational challenges.

Every company that deals with payments deals with fraud. The question is not whether fraud will occur on your system, but rather how much of it you can detect and prevent. If a payments company flags too many transactions as fraudulent, then real transactions might accidentally get flagged as well. But if you don’t reject enough of the fraudulent transactions, you might not be able to make any money at all. Because fraud detection is such a difficult optimization problem, it is a good fit for machine learning. Today’s guest Michael Manapat works on machine learning fraud detection at Stripe. This conversation explores aspects of both data science and data engineering. Michael seems to benefit from having a depth of knowledge in both aspects of the data pipeline, which made me question whether data science and data engineering are roles that an engineering organization wants to separate. This is the third in a series of episodes about Stripe engineering. Throughout these episodes, we’ve tried to give a picture for how Stripe’s engineering culture works. We hope to do more experimental series like this in the future. Please give us feedback for what you think of the format by sending us email, joining the Slack group, or filling out our listener survey. All of these things are available on softwareengineeringdaily.com.

If you are building a service that processes payments, your software architecture has a lot of requirements. Not only do you need to be highly available, consistent, and fast–you need to be PCI compliant. In this episode, we explore the infrastructure of Stripe with Evan Broder, who has been with the company for five years. Stripe started as a small payments company catering to developers with a monolithic code base. Some of those aspects of Stripe have changed, and others have stayed the same. In our last episode, we covered how observability works at Stripe. In this episode, we explore what is being observed–the actual infrastructure itself, and how different engineers are organized around managing the infrastructure. In tomorrow’s episode, we’ll talk to Michael Manapat, about Stripe’s machine learning pipeline for detecting and preventing fraudulent transactions. Throughout these episodes, you will get a sense for how Stripe’s engineering culture works. We hope to do more experimental series like this in the future. Please give us feedback for what you think of the format by sending us email, joining the Slack group, or filling out our listener survey. All of these things are available on softwareengineeringdaily.com.

Observability allows engineers to understand what is going on inside their systems. In its most raw form, observability comes from log data. Modern systems have many layers of logs–virtualized cloud infrastructure, container orchestration, the container runtime itself, and the application logic running within the container. With all of these layers, it is not practical for a developer to have to sift through layers of logs every time a bug occurs in production, or a deployment fails integration tests. Higher level observability tools include charts, distributed tracing tools, and monitoring services. With proper observability, developers can save time during incident response. Day-to-day software development becomes safer and more comfortable. Stripe is a payments company for developers. This episode is the first in a series of episodes profiling different aspects of the company. Our guest Cory Watson leads the observability team at Stripe. In subsequent episodes, we will explore infrastructure and machine learning at Stripe. Throughout these episodes, you will get a sense for how Stripe’s engineering culture works. We hope to do more experimental series like this in the future. Please give us feedback for what you think of the format by sending us email, joining the Slack group, or filling out our listener survey. All of these things are available on softwareengineeringdaily.com.

Donald Trump has either alarmed or excited everyone in the engineering community. Some of the debates are based around innovation. Opponents to Trump say that his anti-immigration policies will reduce the innovation that relies on H1-B visas. Supporters of Trump say that his anti-regulation policies will unlock innovation that is restricted by bureaucracy. Other debates are around philosophy and free speech. Opponents to Trump say that his views on women are retrograde and that he promotes racism and white nationalism. Supporters of Trump say that he is brave enough to say what people are really thinking, and that he is truly independent from the polarized, ossified political system. Software Engineering Daily is a show about the world through the lens of the software engineer, and today our world is being shaped by Donald Trump. We needed to do an episode on this subject because so many of my day-to-day conversations with other engineers were becoming about Trump. Brad Taylor is a senior engineer with Optimizely who has formed Tech Stands Up, a grassroots movement giving a voice to the rapidly growing concerns about the Trump administration’s policies affecting the tech community and its users. We had a stimulating conversation about the tradeoffs of a Trump administration from an engineer’s perspective. Don’t worry–this is not becoming a show about politics. Tech Stands Up: Pi Day Rally Tickets TechStandsUp.org

Making the right engineering choices in today’s wide landscape of cloud technologies is hard. Predicting the future in order to invest in companies in this space has the same level of complexity. The cost of cloud computing is going down but the volume of total required space and processing power is going up. The open source community is growing and improving but people are increasingly willing to buy software that will save them time. Capital is flowing into Silicon Valley at a faster rate than can be sensibly absorbed by the number of quality companies. A decent product won’t have trouble raising money. But a decent investor has to choose wisely among the huge selection of available opportunities. Lenny Pruss works at Amplify Partners where he is currently focused on the cloud native space. In this episode, we talk about what cloud native means and how to navigate the complex landscape, whether you are an engineer or an investor. Cloud Native Landscape Project

Command Query Responsibility Segregation (CQRS) is a powerful concept that has the potential to make for reliable and maintainable systems. It is also broadly misunderstood and means different things to different people. Derek Comartin learned about the idea after viewing some talks by Greg Young and has since successfully applied the approach with great success and it has transformed the way he views features, business requirements, and dependencies. The result is a system that is easier to maintain and faster to enhance. Among his key lessons are that slices are better than layers, mediators improve dependency management, and cohesion is better applied to business concerns than to technical ones. In this episode, Derek joins Dave Rael for a conversation about CQRS and applying it to make your code your own and to separate it from technical concerns in order to make your software development operation work better and faster.

Machine learning has improved both in tools and accessibility. Frameworks like TensorFlow create the right abstractions for developers to work efficiently. Educational programs like Metis and Insight Data Science provide a place for developers to learn these tools. As a result, artificial intelligence is becoming easier to develop and more widespread. Rumman Chowdhury works on artificial intelligence at Accenture. Before her current role, she taught data science at Metis. In this episode, we talk about the current state of artificial intelligence, from the tools available to the long term implications–such as the robot tax recently proposed by Bill Gates.

Load testing measures performance of a system undergoing a large volume of requests. Before an application is pushed to production, engineers will often load test their software to ensure it is resilient in the face of high traffic. As web applications have changed, the requirements around load testing have changed as well. External APIs, internal undocumented APIs, and proprietary databases are black boxes that you might not be able to test reliably with unit tests or integration tests. Having an end-to-end load testing system can provide a measure of insurance against unknown unknowns before users start engaging with a production version. Mark Gilbert works on performance engineering at Apica, and he joins the show to discuss how load testing software is built and when engineers should use it. Full disclosure: Apica is a sponsor of Software Engineering Daily.

There is too much mobile engineering work to be done and not enough mobile engineers. As a result, a talented mobile engineer will often make more money than a similarly talented web developer. There are many other disconnects between the world of mobile engineering and the world of backend and web development. We have reported on web development far more than mobile on Software Engineering Daily, and, I wanted to get a holistic view of the mobile ecosystem. Nathan Esquenazi is a co-founder of CodePath, which is a continuing education program for professional engineers and designers. There was much to explore–from the economics of mobile, to the state of cross platform, to the increasingly blurry line between mobile and IoT.

Thursday February 23rd was a big day in security news: details were published about the Cloudbleed bug, which leaked tons of plaintext requests from across the Internet into plain view. On the same day, the first collision attack against SHA-1 was demonstrated by researchers at Google, foretelling the demise of SHA-1 as a safe hashing function. What does this mean for the average engineer? What are the implications for regular internet users? Haseeb Qureshi interviews Max Burkhardt, a security researcher at Airbnb, to get to the bottom of what exactly happened, what it means, and how it affects the security of web applications.

Manufacturing electronics is not a simple task and because of its complexity it is also quite expensive. Companies like Apple are able to pull their costs down through economies of scale. Since Apple is always placing huge bulk orders on chips, sensors, and other components, the company can get lower prices for those components than an individual hardware hacker that wants to build a prototype. Herein lies the opportunity for Tempo Automation, a company that allows rapid prototyping of electronics. By aggregating demand for electronics prototyping, Tempo Automation can offer competitive pricing and speed to anyone who needs a hardware prototype. I visited Tempo Automation and was given a tour of the production floor by CEO Jeff McAlvay. He showed me the huge machines that are used to build hardware prototypes and explained how they work in serial to produce prototypes. He also mentioned that anyone can schedule a tour of Tempo Automation’s factory by going to their website, and I highly recommend it. After the tour, Jeff and I sat down for an interview about what Tempo Automation is building and their plans for the future.

When an engineer is offered a job a tech company, their compensation is often partly in cash and partly in equity–shares of the company. How should an engineer evaluate that offer? How should they negotiate? In the world of equity compensation, costly and avoidable mistakes are routine, and this hurts both companies and employees. Josh Levy was on Software Engineering Daily previously to talk about the Amazon Web Services open guide, which was one of the most popular episodes we have ever had. In this episode, Josh and I are joined by Joe Wallin, a lawyer who has been involved in startups for many years. Josh and Joe are the authors of the The Open Guide to Equity Compensation, which is a resource designed to clear up the confusion around stock, options, and fundraising. It’s a tremendously useful and concise overview of what an engineer or founder needs to know when it comes to equity financing.

Parse was a backend as a service company built in 2011 before being acquired by Facebook in 2013. Building a backend as a service for developers requires walking a thin line between giving engineers lots of control and preventing those engineers from shooting themselves in the foot. While she was at Parse, Charity Majors learned about the operational burdens of managing a service with high uptime requirements and deeply technical edge cases that could take down a user’s entire system. Charity took the lessons in systems engineering that she learned at Parse and cofounded Honeycomb.io, a service for observability and monitoring. Honeycomb is described as a tool for your systems like an IDE is to your code. Parse was eventually shut down because the service did not have a place in the strategic plans of Facebook. Charity and I also discussed the lessons learned from how the Parse acquisition panned out–a useful conversation for anyone who is considering selling a company or acquiring a startup.

When an application is using all of its available resources, that application needs to be scaled. Scaling an application means giving it more resources–typically servers. Autoscaling is an engineering practice where an application is automatically given more or less resources based on how healthy the application performance is at a given time. Applications on Heroku have access to autoscaling. Heroku users don’t need to worry about provisioning new servers manually because the platform does it for them. In this episode, we explore how Heroku built autoscaling. Andrew Gwozdziewycz [@apgwoz] is an operational experience engineer with Heroku. As he describes, autoscaling requires frequent health checks of an application. Since thousands of applications are running on Heroku, a metrics pipeline using Kafka and Cassandra supports the high volume of health check data. That data feeds into the decision process for when an application needs to scale. Full disclosure: Heroku is a sponsor of Software Engineering Daily.

In the mid 90s, data warehousing might have meant “using an Oracle database.” Today, it means a wide variety of things. You could be stitching together a big data pipeline using Kafka, Hadoop, and Spark. You could be using managed tools like BigQuery from Google. How did we get from the simple days of Oracle databases to the wealth of options available today? Mark Rittman writes and podcasts about data engineering and data warehousing on his site Drill to Detail. Today, we explore the past, present, and future of data warehousing and touch on many of the trends that have been explored in recent episodes of Software Engineering Daily. Google BigQuery, and Why Big Data is About to Have Its Gmail Moment

At the first Software Engineering Daily Meetup, the speakers explored a range of topics. A few weeks ago, we published Courtland Allen’s talk about how to build a small software business. In today’s episode, we are publishing Haseeb Qureshi’s talk, called “Everything That Rises Must Converge: Why Engineers Disagree About Everything (And Why Fraudsters Do Too).” This talk explores philosophy, poker, software engineering, fraud, and a basket of other topics. These topics might sound like a random collection, but Haseeb currently works as an engineer on Airbnb’s Risk team and he is the author of How To Be a Poker Player, which is a book on the philosophy of poker. The audio is taken from the Meetup and we cleaned it up as best we could. We will do a better job with the audio from the next Meetup–which is Thursday, March 9th at Galvanize! Check out our Meetup page for more details.

Security vulnerabilities are an important concern in systems. When we specify that we want certain information hidden, for example our phone number or our date of birth, we expect the system to hide the information. However, this doesn’t always happen due to human error in the code because programmers have to write checks and filters across the program. In this episode, Edaena Salinas interviews Jean Yang, Assistant Professor at the Computer Science Department at Carnegie Mellon, who presents Jeeves, a language that allows programmers to specify security policies more intuitively, making it harder to leak information that is meant to be protected. Jean explained how Jeeves was implemented and how it can be used. We also talked about what it takes to bring research concepts from academia to the industry and at the end we had a very interesting conversation on how to educate a broader audience on the importance of security. Jean was also named one of the 35 innovators under 35 by MIT Technology Review.

Expectations for mobile apps have gone up steadily since the iPhone was released. But the choice of databases built for mobile apps has remained limited mostly to SQLite. RealmDB was created as a new option for mobile developers on iOS, Android, or any other mobile platform. Realm is not just a database. It is a database platform, offering a variety of systems such as authentication, sync, and an event framework. In this interview with RealmDB VP of engineering Brian Munkholm, he explains why we need not only a new mobile database type, but an entire database platform. We talk about the business of mobile, IoT, cloud, and cross platform. It’s a wide ranging conversation, but also deeply technical.

Online advertising is a system of transactions that involve many different players. The user visits a publisher’s website; the publisher notifies an exchange that the user is on the website; the exchange presents an opportunity to a marketplace that can buy that opportunity to show the end user the ad. And this a simple example. The transactions in online advertising are as opaque and rife with fraud as Wall Street–but less regulated. Blockchain technology presents an opportunity to bring more transparency to the advertising ecosystem using a shared ledger. Ken Brook is the CEO of VidRoll, a video advertising company. Using his experience in the adtech business, Ken is working on Adchain, a shared ledger for adtech. In this episode, we explore adtech, blockchains, and Adchain. Russian Cyberforgers Steal Millions a Day with Fake Sites

The impact of artificial intelligence on our everyday lives will be so profound that our modern institutions will change completely. Employment, government, romance, social norms–all of these things will be upended. To see the signs of this coming, you no longer have to read science fiction. Every week, there are blog posts, news stories, and videos chronicling our strange, exciting time. Rob May is an investor in artificial intelligence companies and is CEO of the AI company Talla. Every week, he puts out the Technically Sentient newsletter, a compilation of the best pieces of information about AI over the past week. Each newsletter also contains a short essay by Rob in which he gives a big picture perspective on what he is seeing in the AI space. This was an illuminating conversation with Rob about the implications of artificial intelligence and the topics he writes about in Technically Sentient. I first saw Rob speak at the Launch Scale Festival, which Jason Calacanis puts on as a free event for people who have started companies.

If you wanted to build a machine learning model to understand human health, where would you get the data? A hospital database would be useful, but privacy laws make it difficult to disclose that patient data to the public. In order to publicize the data safely, you would have to anonymize it, so that a patient’s identity could not be derived from data about that patient–and true anonymization is notoriously difficult. In every industry where privacy is a concern there is a similar challenge. If there is no place with public data sets, there is no place where the machines can go to learn. The possible machine learning applications that we can build are limited by the data sets that are available. Auren Hoffman started his company SafeGraph to unlock data sets so that machine learning algorithms can learn from that data. In this episode, we talk about the machine learning landscape in both the short- and long-term time horizons. We also discussed some of Auren’s strategies for building companies, which have been crucial for me in thinking about how to build Software Engineering Daily. Where Should Machines Go to Learn? Quoracast Episode

Machine learning frameworks like Torch and TensorFlow have made the job of a machine learning engineer much easier. But machine learning is still hard. Debugging a machine learning model is a slow, messy process. A bug in a machine learning model does not always mean a complete failure. Your model could continue to deliver usable results even in the presence of a mistaken implementation. Perhaps you made a mistake when cleaning your data, leading to an incorrectly trained model. It is a general rule in computer science that partial failures are harder to fix than complete failures. In this episode, Zayd Enam describes the different dimensions on which a machine learning model can develop an error. Zayd is a machine learning researcher at the Stanford AI Lab, so I also asked him about AI risk, job displacement, and academia versus industry. Why ML is hard

Data scientists need flexible interfaces for displaying and manipulating data sets. Data engineers need to be able to visualize how their data pipelines wire together databases and data processing frameworks. DevOps engineers need dashboards to understand their monitoring data at a high level. All of these programmers are building data applications. Data applications let us visualize and manipulate data sets effectively. In today’s episode, Dave King joins the show to describe the growing importance of data applications. Many knowledge workers use flexible tools like spreadsheets and business intelligence applications. But when you build a domain specific data application for a knowledge worker, you can unlock a higher degree of leverage for that knowledge worker. We discuss data applications and the future of knowledge work in this episode with Dave King. Full disclosure: Exaptive is a sponsor of Software Engineering Daily.

Most tech companies are moving toward a highly distributed microservices architecture. In this architecture, services are decoupled from each other and communicate with a common service language, often JSON over HTTP. This provides some standardization, but these companies are finding that more standardization would come in handy. At the ridesharing company Lyft, every internal service runs a tool called Envoy. Envoy is a service proxy. Whenever a service sends or receives a request, that request goes through Envoy before meeting its destination. Matt Klein started Envoy, and he joins the show to explain why it is useful to have this layer of standardization between services. He also gives some historical context for why Envoy was so helpful to Lyft.

Infrastructure is a term that can mean many different things: your physical computer, the data center of your Amazon EC2 cluster, the virtualization layer, the container layer–on and on. In today’s episode, podcasters Chris Wahl and Ethan Banks discuss the past, present, and future of infrastructure with me. Ethan and Chris host Datanauts, a podcast about infrastructure. In each episode, Datanauts goes deep on a topic such as networking, serverless, or OpenStack. As someone who hosts a similar podcast, I find it entertaining and educational to hear their points of view on a regular basis. If you like Software Engineering Daily, you might like Datanauts. And if you like Datanauts, you will love this episode of Software Engineering Daily.

Deep learning uses neural networks to identify patterns. Neural networks allow us to sequence “layers” of computing, with each layer using learning algorithms such as unsupervised learning, supervised learning, and reinforcement learning. Deep learning has taken off in the last few years, but it has been around for much longer. Adam Gibson founded Skymind, the company behind Deeplearning4j. Deeplearning4j is a distributed deep learning library for Scala and Java. It integrates with Hadoop and Spark, and is specifically designed to run in business environments on distributed GPUs and CPUs. Adam joins the show today to discuss the history and future of deep learning.

Data science is typically done by engineers writing code in Python, R, or another scripting language. Lots of engineers know these languages, and their ecosystems have great library support. But these languages have some issues around deployment, reproducibility, and other areas. The programming language Golang presents an appealing alternative for data scientists. Daniel Whitenack transitioned from doing most of his data science work in Python to writing code in Golang. In this episode, Daniel explains the workflow of a data scientist and discusses why Go is useful. We also talk about the blurry line between data science and data engineering, and how Pachyderm is useful for versioning and reproducibility. Daniel works at Pachyderm, and listeners who are more curious about it can check out the episode I did with Pachyderm founder Joe Doliner.

Engineering managers face a different set of problems than engineers themselves. Whether they are hiring new employees, firing underperformers, or guiding a team of existing engineers, engineering management is all about people. Empathy is paramount because management is like being a professional therapist for the members of your team. Mike Borozdin has been managing engineers for more than a decade and blogs about engineering management. In this episode, we cover the core responsibilities of an engineering manager and some strategies around how to do it right. This episode is useful for anyone looking to improve their relationship with their manager, move into management, or improve as a manager.

Open source software is publicly available code that is worked on in the open by large crowds of developers. Almost all new software today uses some open source software in its code. But most people never contribute to open source themselves. Some people would love to get involved in open source, but they don’t know how to get started. Shubheksha Jalan found herself in this position. In her early attempts to break into open source, she failed to get much traction and found the open source community hostile to newcomers. She would fix beginner issues, but had trouble figuring out how to make more complex contributions. Eventually she found success and wrote about it. In this episode she takes us through that journey. If you are wondering how to get into open source, this episode is for you.

Giphy is a search engine for gifs, the short animated graphics that we see around the Internet. Giphy is also a creative platform where people create new gifs. Every search engine requires the construction of a search index, which is a data structure that responds to search queries efficiently. Since Giphy is a search engine for graphics, there is almost no text inherently associated with the each document. Giphy uses a pipeline of different labeling techniques in order to make a gif indexable by the search engine. In my conversation with Giphy CTO Anthony Johnson, we discuss how to scale a search engine, why Giphy needs to build new techniques for image processing, how human labeling for machine learning is evolving, and the future of Giphy–both as a creative medium and an advertising platform. This was an exciting and wide-reaching interview.

Cloud computing has pushed computation away from our own private servers and into virtual machines running on a data center. In the world of cloud computing, processing is centralized in these data centers, and our smartphone and laptop application performance suffers from having high latency between the client and the cloud server. As machine learning proliferates, the current model of cloud computing will become too slow. A small difference in the time it takes to refresh a machine learning model for a drone or car could be the difference between life and death. Computation will move to the edge. The same drones, cars, and IoT devices that need their models updated quickly will form a peer-to-peer network with which to distribute time-sensitive tasks. One bellwether for this real time peer-to-peer network might be Uber’s Ringpop, a fault-tolerant application layer sharding system. In such an edge computing model, your device could federate a complex request out to other nearby devices that have spare processing capacity, pay for those compute cycles using Bitcoin, and receive a response without any request to a centralized cloud server. The cloud servers would still be around, but they would be responsible for doing offline computation across large data sets. This prediction was described by Peter Levine, a partner at Andreessen Horowitz in his talk “The End of Cloud Computing”. In this episode, Peter discusses the pressures that our pushing toward edge computing and away from the cloud.

What is the relationship between your brain and your conscious experiences? This is is the fundamental question of the work of Donald Hoffman, a professor of computer science and cognitive science at UC Irvine. When Hoffman was a child, he wondered whether there was a cognitive dividing line between humans and machines, and that curiosity has driven him to his current work–building a mathematical framework which we can use to model consciousness. Humans would like to believe that evolution selects for traits that allow us to understand the world as it is. From his experimental simulations, Hoffman has shown the opposite–that evolution pushes us towards a mistaken version of reality. For listeners who are interested in theories about whether we live in a simulation, this episode is for you. If you are skeptical of the simulation theory then this episode will also be useful for you, as Hoffman gives one of the most nuanced, comprehensive explanations of reality that I have heard. TED Talk The Case Against Reality

Software developers succeed by combining technical ability, communication skills, and well-reasoned philosophies to craft information systems. Where this podcast focuses on the information systems, Developer On Fire focuses on the engineers who built them. Dave Rael started his podcast as a way to overcome fear and access an entrepreneurial side of himself. In his interviews with prominent engineers, Dave identifies patterns and strategies that have enabled his guests to succeed. He also finds the human side–namely the fallibilities and failures–of legendary developers, which makes it easier to imagine ourselves accomplishing the feats of his guests.

Back in 2008, the range of tools that engineers could use to connect computer systems together were getting quite good. Cloud computing was democratizing access to servers. But the telephony ecosystem was still inaccessible to the average developer. If you needed your program to make a phone call and connect a user to a customer service representative, there was no easy way to do that. Twilio was started to make it easy for developers to connect to telephone systems using simple API calls. This has unlocked many important use cases: from Uber’s communication systems to the widespread adoption of 2-factor authentication. In this episode, Twilio VP of product management Pat Malatack joins the show to explain how the company builds and scales the telephony systems that underpin applications which we use every day. We also talked about how Twilio’s culture shapes how engineering proceeds at the company. Full disclosure: Twilio is a sponsor of Software Engineering Daily.

Engineers today have a variety of career options. You could go work for a large corporation, you could raise money and start a startup, you could freelance and move from job to job with freedom–or you could start a business with the goal of quickly becoming profitable. Courtland Allen was a guest on Software Engineering Daily a few months ago, when he discussed Indie Hackers, a platform he built to share the stories of engineers building business on their own and making money. We only touched the tip of the iceberg in our conversation, so I was excited to invite him to the first Software Engineering Daily Meetup, which occurred earlier this month. Today we are republishing his talk, and I would love to hear your feedback on this format. We will be experimenting more with new hosts and formats throughout 2017, and if you have ideas for the show or you are interested in hosting a show, please send me an email. Also–the first Software Engineering Daily Meetup was fantastic–there were ~200 people showing up so we may have to cap attendance on the next one. Please join the Meetup group if you are interested, and we will let you know when we schedule our next event. Courtland Allen slides in PDF.

Many elderly people live with unhealthy levels of isolation. Social isolation is a problem for anybody, but younger people can use technology to alleviate their isolation with tools like Skype and Facebook. How can we bridge the generational gap and give elderly people access to the same technological tools that younger people find easy to use? Voice interfaces are an important new medium for communicating with computers. Amazon and Google are at the forefront of the voice interface movement, but many other devices are emerging. One of these devices is Elliq, from Intuition Robotics. Elliq consists of a microphone voice interface that sits on a table next to a small tablet computer. An elderly person can talk to the voice interface and have it start a voice call with someone on Skype, or see new Facebook photos or YouTube videos. The discussion of Elliq’s hardware is as interesting as that of its backend software. As Itai Mendelsohn of Intuition Robotics explains, Elliq got to market very quickly as a result of the company’s adoption of high-level managed services, like Firebase and Google’s managed machine learning. ELLIQ – The active aging companion

Whatever engineering problem you have right now, the solution is probably not to write a new programming language. But sometimes it does make sense. JetBrains makes IDEs–the interactive development environments that many people code in, like IntelliJ and Webstorm. And all of these IDEs are written in Java. So the JetBrains team is very familiar with Java and the JVM. Since JetBrains spends so much time working on tooling for developers, they also have an intimate understanding of the problems that developers encounter. Kotlin was developed as an alternative to JVM languages such as Java and Scala. In this episode, Hadi Hariri from JetBrains explains why the company decided to build a new JVM language. We discuss many of its features, such as safety, concision, and its ability to compile to JavaScript. Why Kotlin is My Next Programming Language: Ode to the Language You’ve Never Heard Of, by Mike Hearn YouTube: GOTO 2016 Conference, Kotlin – Ready for Production, Hadi Hariri

Translation is a classic problem in computer science. How do you translate a sentence from one human language into another? This seems like a problem that computers are well-suited to solve. Languages follow well-defined rules, we have lots of sample data to train our machine learning models. And yet, the problem has not been solved–largely because languages don’t always follow rules. We have idioms and subtle contextual clues that make it hard to provide a computer with hard and fast rules for translation. Unbabel is a company whose solution to translation puts a human in the loop to correct the error-prone translations that computers often make. In this episode, Vasco Pedro joins the show to explain Unbabel’s approach to translation, its technology stack, and the business applications for translation.

The goals of microservices are the same as what we have pursued in software engineering for decades: isolation, decoupling, maintainability, scalability. The reason that we use the term microservices is not because we have a completely different idea of what a service is than we used to. We use the term microservices because we are signaling that we need to achieve these architectural goals in a different way than we needed to 10-15 years ago. Markus Eisele is a developer advocate at Lightbend. He joins the show to discuss how enterprises are moving from monolithic architectures to microservices architectures, which has been touched on in previous shows. Lightbend makes a framework called Lagom, that suggests a opinionated strategy for moving towards microservices using message passing, CQRS, and other patterns that we explore. Reactive Microservices Architecture

Building a mobile app requires developers to build a separate version for Android and iOS. The approval process for app stores makes it difficult to deploy quickly and iterate in small batches for developers who are making native apps. These frictions cause fewer developers to write mobile apps than we would have if the smartphone platform was unified. Since the early days of the smartphone, different cross platform solutions have been tried, and many developers got burned after they implemented an app that was supposed to work on iOS and Android, when the performance ended up being poor on both. Today, the cross-platform solutions are finally getting good with frameworks like React Native and Ionic. Ionic allows developers to write apps in AngularJS and deploy them to both Android and iOS. In this episode, Adam Bradley from Ionic gives a brief history of cross-platform apps, and explains what is different today–and why cross platform apps are exploding in popularity, particularly in large enterprises.

Urbit is a completely new way of looking at computing. Every user gets a personal server, which runs your apps, wrangles your connected devices, and defines your secure identity. Your urbit presents your whole digital life as a single web service. Urbit feels foreign and confusing for those of us coming from the traditional web because the normal paradigm is to iterate and paper over the problems of the current platform with new things built on top. Curtis Yarvin, the creator of Urbit, argues that the current model is too fundamentally broken for that to work. As he says: “the Internet as a client-server network has won. The Internet as a peer-to-peer network has failed.” This sounds like yet another quirky, overambitious developer side project–but Urbit has serious legs. The github repo has had 51 committers over its four years of activity. Last year, a public crowdsale of Urbit address space raised more than $200,000. Peter Thiel was an early investor in the project, perhaps partly due to the combination of persistence, technical skill, and unusual opinions of Curtis Yarvin. In this episode, Curtis and Galen Wolfe-Pauly join me for a conversation about Urbit–its strange computing platform and the contrarian philosophies that drive its creators.

Over the past few years, React has become the most popular front end JavaScript framework. As React has matured, the open source community around React has identified areas for improvement. Since React itself is too mature to refactor completely, new projects have been started to take the best aspects of React and start from scratch. Inferno is an extremely fast, React-like JavaScript library for building modern user interfaces. Dominic Gannaway is the creator of Inferno and in this episode he joins Caleb Meredith for an interview about Inferno and other React-like JavaScript libraries.

Product managers are responsible for guiding the design and overall functionality of software. The relationship between product managers and engineers is complimentary: a PM is viewing the product from a perspective that is closer to the customer, so the PM often has the responsibility of navigating high-level tradeoffs in the functionality of a product. Suzie Prince is the head of product at ThoughtWorks and she joins the show to discuss how she navigates tradeoffs as a PM. We also explored communication strategies for how PMs can work productively with engineers, and the overall product development process. Full disclosure: SnapCI is a sponsor of Software Engineering Daily.