Greatest Hits Archives - Software Engineering Daily

When software is performing suboptimally, the programmer can use a variety of tools to diagnose problems and improve the quality of the code. A profiler is a tool for examining where a program is spending time. Every program consists of a set of different functions. These functions call each other. The total amount of time The post Profilers with Julia Evans appeared first on Software Engineering Daily.

Applications of artificial intelligence are permeating our everyday lives. We notice it in small ways–improvements to speech recognition; better quality products being recommended to us; cheaper goods and services that have dropped in price because of more intelligent production. But what can we quantitatively say about the rate at which artificial intelligence is improving? How The post OpenAI: Compute and Safety with Dario Amodei appeared first on Software Engineering Daily.

Self-driving, electric cars will someday outnumber traditional automobiles on the road. As transportation becomes autonomous, it is hard to imagine an industry that will not be affected by the downstream effects of this change. These cars will likely be managed by fleet operators like Lyft and Uber. We will need fewer cars, and the amount The post Autonomy with Frank Chen appeared first on Software Engineering Daily.

When a user takes a ride on Uber, the app on the user’s phone is communicating with Uber’s backend infrastructure, which is writing to a database that maintains the state of that user’s activity. This database is known as a transactional database or “OLTP” (online transaction processing). Every active user and driver and UberEATS restaurant The post Uber’s Data Platform with Zhenxiao Luo appeared first on Software Engineering Daily.

A sample of the human voice is a rich piece of unstructured data. Voice recordings can be turned into visualizations called spectrograms. Machine learning models can be trained to identify features of these spectrograms. Using this kind of analytic strategy, breakthroughs in voice analysis are happening at an amazing pace. Rita Singh researches voice at The post Voice with Rita Singh appeared first on Software Engineering Daily.

Mesos is a system for managing distributed systems. The goal of Mesos is to help engineers orchestrate resources among multi-node applications like Spark. Mesos can also manage lower level schedulers like Kubernetes. A common misconception is that Mesos aims to solve the same problem as Kubernetes, but Mesos is a higher level abstraction. Ben Hindman The post Cluster Schedulers with Ben Hindman appeared first on Software Engineering Daily.

Technology is pushing us rapidly toward a future that is impossible to forecast. We try to imagine what that future might look like, and we can’t help having our predictions shaped by the media we have consumed. 1984, Terminator, Gattaca, Ex Machina, Black Mirror–all of these stories present a dystopian future. But if you look The post Technology Utopia with Michael Solana appeared first on Software Engineering Daily.

Google’s central system for managing to compute resources is called Borg. On Borg, millions of Linux containers process a wide variety of workloads. When a new application is spun up, Borg provides that application with the resources it needs. Workloads at Google usually fall into one of two distinct categories: long-running application workloads (such as The post Google Cluster Evolution with Brian Grant appeared first on Software Engineering Daily.

Machine learning tools are rapidly maturing. TensorFlow gave developers an open source version of Google’s internal machine learning framework. Cloud computing provides a cost effective, accessible way of training models. Edge computing allows for low latency deployments of models. But even if you are a kid with a laptop who has learned all the machine The post SafeGraph with Auren Hoffman appeared first on Software Engineering Daily.

“The Federal Reserve System is fraudulent. Whatever its stated purpose, its effective purpose is to create a mechanism of deficit spending by politicians, through the insidious invisible taxation of monetary debasement (aka inflation).” These are the words of Erik Voorhees, the CEO of crypto financial exchange ShapeShift. Long before he started ShapeShift, Erik was opposed The post ShapeShift with Erik Voorhees appeared first on Software Engineering Daily.

Cryptocurrency speculation has pulled in a large population of people who do not know what they are investing in. If you hear about an investment of $1000 turning into $1M, it’s tempting to get sucked in yourself. For most of these everyday people, the game is completely rigged. A large percentage of market activity is The post Crypto Pump and Dumps with Bruno Skvorc appeared first on Software Engineering Daily.

Joseph Bonneau is co-author of Bitcoin and Cryptocurrency Technologies, a popular textbook. At NYU, he works as an assistant professor exploring cryptography and security. His YouTube lessons teaching Bitcoin have hundreds of thousands of views. His material offers clear explanations of how Bitcoin works. Since Joseph has a clear understanding of the objective facts around The post Bitcoin’s Future with Joseph Bonneau appeared first on Software Engineering Daily.

Dogecoin was started in 2013 as a joke. Jackson Palmer forked Bitcoin and created his cryptocurrency as a play-off of the “doge” meme. The currency became popular as a means of Reddit users “tipping” each other. If I made a comment on Reddit that you liked, you might send me some Dogecoin. This use case The post Dogecoin with Jackson Palmer appeared first on Software Engineering Daily.

Apache Spark is a system for processing large data sets in parallel. The core abstraction of Spark is the resilient distributed dataset (RDD), a working set of data that sits in memory for fast, iterative processing. Matei Zaharia created Spark with two goals: to provide a composable, high-level set of APIs for performing distributed processing; The post Spark and Streaming with Matei Zaharia appeared first on Software Engineering Daily.

When Box started in 2006, the small engineering team had a lot to learn. Box was one of the earliest cloud storage companies, with a product that allowed companies to securely upload files to remote storage. This was two years before Amazon Web Services introduced on-demand infrastructure, so the Box team managed their own servers, The post Scaling Box with Jeff Quiesser appeared first on Software Engineering Daily.

Your friends from college are asking you how to buy Bitcoin. Your mom is emailing you articles about the benefits of decentralized peer-to-peer networks. Your shoe shiner is telling you to buy XRP. It is 2018, and cryptocurrencies have become a daily part of news headlines. The general public may not understand how this technology The post Tether, Ripple, and Blockchain Reporting with Matt Leising appeared first on Software Engineering Daily.

Kubernetes has become the standard way of deploying new distributed applications. Most new internet businesses started in the foreseeable future will leverage Kubernetes (whether they realize it or not). Many old applications are migrating to Kubernetes too. Before Kubernetes, there was no standardization around a specific distributed systems platform. Just like Linux became the standard The post The Gravity of Kubernetes appeared first on Software Engineering Daily.

Kubernetes has become the standard system for deploying and managing clusters of containers. But the vision of the project goes beyond managing containers. The long-term goal is to democratize the ability to build distributed systems. Brendan Burns is a co-founder of the Kubernetes project. He recently announced an open-source project called Metaparticle, a standard library The post Kubernetes Vision with Brendan Burns appeared first on Software Engineering Daily.

You are requesting a car from a ridesharing service such as Lyft. Your request hits the Lyft servers and begins trying to get you a car. It takes your geolocation, and passes the geolocation to a service that finds cars that are nearby, and puts all those cars into a list. The list of nearby The post High Volume Distributed Tracing with Ben Sigelman appeared first on Software Engineering Daily.

Originally published January 6, 2016 “The best computer science is the kind where the theory is inspired by some practical problem, you develop a better theoretical understanding of what you want to do, and that feeds back into better practice.” Brian Kernighan is a professor of computer science at Princeton University and the author of The post Language Design with Brian Kernighan Holiday Repeat appeared first on Software Engineering Daily.

Originally published November 18, 2015 “The playing field has never ever been more leveled – that means everything you don’t build is your choice not to build it.” Seth Godin is a writer, speaker, and entrepreneur. He is the author of many books, including most recently, What To Do When It’s Your Turn. Questions How The post Software and Entrepreneurship with Seth Godin Holiday Repeat appeared first on Software Engineering Daily.

Originally published November 10, 2015 “The cloud as an environment – I had thought it was a purely utilitarian kind of thing. What I realized is that it’s a fascinating centralized repository of computation.” Wolfram Research makes computing software powered by the Wolfram language, a knowledge-based programming language that draws from symbolic and functional programming The post Knowledge-Based Programming with Stephen Wolfram Holiday Repeat appeared first on Software Engineering Daily.

Originally published November 17, 2015 “Changing anything changes everything.” Technical debt, referring to the compounding cost of changes to software architecture, can be especially challenging in machine learning systems. D. Sculley is a software engineer at Google, focusing on machine learning, data mining, and information retrieval. He recently co-authored the paper Machine Learning: The High The post Machine Learning and Technical Debt with D. Sculley Holiday Repeat appeared first on Software Engineering Daily.

There is a quote from Jeff Bezos: “70% of the work of building a business today is undifferentiated heavy lifting. Only 30% is creative work. Things will be more exciting when those numbers are inverted.” That quote is from 2006, before Amazon Web Services had built most of their managed services. In 2006, you had The post Run Less Software with Rich Archbold appeared first on Software Engineering Daily.

Automation is changing the labor market. To automate a task, someone needs to put in the work to describe the task correctly to a computer. For some tasks, the reward for automating a task is tremendous–for example, putting together mobile phones. In China, companies like FOXCONN are investing time and money into programming the instructions The post Training the Machines with Russell Smith appeared first on Software Engineering Daily.

A popular software application serves billions of user requests. These requests could be for many different things. These requests need to be routed to the correct destination, load balanced across different instances of a service, and queued for processing. Processing a request might require generating a detailed response to the user, or making a write The post High Volume Event Processing with John-Daniel Trask appeared first on Software Engineering Daily.

As the gig economy grows, that growth necessitates innovations in the online infrastructure powering these new labor markets. In our previous episodes about Uber, we explored the systems that balance server load and gather geospacial data. In our coverage of Lyft, we studied Envoy, the service proxy that standardizes communications and load balancing among services. The post Fiverr Engineering with Gil Sheinfeld appeared first on Software Engineering Daily.

Imagine that you are a lawyer. Your work involves managing files with dense, technical text. Your co-workers collaborate with you to accomplish a complex goal that can be broken down into smaller pieces. Your work has formal specifications, but there are degrees of freedom in how you express an idea. In all of these ways, The post Legal Technology with Justin Kan appeared first on Software Engineering Daily.

An engineer who wants to start a business using investment capital needs to understand the expectations of investors. The market for the business needs to be huge. The team needs to have a differentiated understanding of the market, or a differentiated product. The CEO needs to have the determination to continue operating the company even The post Early Investments with Semil Shah appeared first on Software Engineering Daily.

The Netflix API is accessed by developers who build for over 1000 device types: TVs, smartphontes, VR headsets, laptops. If it has a screen, it can probably run Netflix. On each of these different devices, the Netflix experience is different. Different screen sizes mean there is variable space to display the content. When you open The post Netflix Serverless-like Platform with Vasanth Asokan appeared first on Software Engineering Daily.

In 2003, Paul Martino co-founded Tribe.net, one of the earliest social networking sites. Tribe had significant traction, with hundreds of thousands of users. In the early 2000s, hundreds of thousands of users was enough traffic to pose a company with engineering challenges. Paul had studied computer science, and was able to use his knowledge of The post Parlaying Failure to Fortune with Paul Martino appeared first on Software Engineering Daily.

In the 1960s, advertising agencies were high-dollar creative producers. A client would come to an ad agency and pay millions of dollars for artistic messaging that would convince a consumer to buy a product. How could you measure the success of these advertising campaigns? Maybe you could see success in the sales data. Maybe people The post Bad Men with Bob Hoffman appeared first on Software Engineering Daily.

In America, the tech companies we focus on are commonly known as FAANG: Facebook, Amazon, Apple, Netflix, Google. We all know what these companies do because they impact our daily lives. In Asia, there are three giant tech companies that have similar scale: Baidu, Alibaba, and Tencent, otherwise known as BAT. Technology within a location The post Analyse Asia with Bernard Leong appeared first on Software Engineering Daily.

Machines understand the world through mathematical representations. In order to train a machine learning model, we need to describe everything in terms of numbers. Images, words, and sounds are too abstract for a computer. But a series of numbers is a representation that we can all agree on, whether we are a computer or a The post Word2Vec with Adrian Colyer appeared first on Software Engineering Daily.

After raising $18 million, social networking startup Yubl made a series of costly mistakes. Yubl hired an army of expensive contractors to build out its iOS and Android apps. Drama at the executive level hurt morale for the full-time employees. Most problematic, the company was bleeding cash due to a massive over-investment in cloud services. The post Serverless Startup with Yan Cui appeared first on Software Engineering Daily.

Quantum computing is based on the system of quantum mechanics. In quantum computing, we perform operations over qubits instead of bits. A qubit is a vector, which can take on many more values than 0 or 1. The technology used to implement quantum computers is advancing such that it has its own Moore’s Law, but The post Quantum Computing with Vijay Pande appeared first on Software Engineering Daily.

Self-driving cars are here. Fully autonomous systems like Waymo are being piloted in less complex circumstances. Human-in-the-loop systems like Tesla Autopilot navigate drivers when it is safe to do so, and lets the human take control in ambiguous circumstances. Computers are great at memorization, but not yet great at reasoning. We cannot enumerate to a The post Self-Driving Deep Learning with Lex Fridman appeared first on Software Engineering Daily.

“Culture fit” is a term that is used to describe engineers that have the right personality for a given company. In the hiring process, “lack of culture fit” is used to turn away engineers who are good enough at coding but just don’t seem right for the company. As today’s guest Ammon Bartram says, “lack The post Culture Fit with Ammon Bartram appeared first on Software Engineering Daily.

The history of computing can be thought of as a series of ideas rather than objects. From Aristotle’s formalization of the syllogism, to Alan Turing’s model for an all-purpose computing machine, to Satoshi Nakamoto’s distributed transaction ledger–these breakthroughs did not come in the form of polished, tangible objects. In fact, the objects which end up The post Computer Logic with Chris Dixon appeared first on Software Engineering Daily.

Search engines run our lives. The path we take to information is dictated by Google, Facebook, Amazon, and other forms of search. Search engines feel objective and truthful, but are built through ongoing experimentation and subjective decision making. That’s what has kept Danny Sullivan writing about search engines for twenty years. The content Google prioritizes, The post Search Engine Land with Danny Sullivan appeared first on Software Engineering Daily.

John Looney spent more than 10 years at Google. He started with infrastructure, and was part of the team that migrated Google File System to Colossus, the successor to GFS. Imagine migrating every piece of data on Google from one distributed file system to another. In this episode, John sheds light on the engineering culture The post Google Early Days with John Looney appeared first on Software Engineering Daily.

Social networks like Facebook and Twitter facilitate interactions between individuals. Every message I send to you on Facebook goes through Facebook’s servers before reaching you. This is known as the client-server model. Since the early days of the internet, engineers have always envisioned a peer-to-peer model, where I could communicate to you directly, without a The post Off-Grid Social Network with Andre Staltz appeared first on Software Engineering Daily.

Humans have now been defeated by computers at heads up no-limit holdem poker. Some people thought this wouldn’t be possible. Sure, we can teach a computer to beat a human at Go or Chess. Those games have a smaller decision space. There is no hidden information. There is no bluffing. Poker must be different! It The post Poker Artificial Intelligence with Noam Brown appeared first on Software Engineering Daily.

CRISPR is a technique for altering the human genome. It might be the most powerful tool for biological modification that we have ever discovered. In this episode, we explore CRISPR: how it works, why it exists in the natural world, and the implications for being able to modify DNA so easily. Geoff Ralston is a The post CRISPR with Geoff Ralston appeared first on Software Engineering Daily.

There are certain experiences when a product solves a problem so thoroughly and elegantly that it lifts a weight off of your shoulders that you didn’t even know was there. Dropbox did this with file storage. Slack did this with group collaboration. Zencastr does this for recording podcasts. Before I used Zencastr to record my The post Zencastr with Josh Nielsen appeared first on Software Engineering Daily.

A new programmer learns to build applications using data structures like a queue, a cache, or a database. Modern cloud applications are built using more sophisticated tools like Redis, Kafka, or Amazon S3. These tools do multiple things well, and often have overlapping functionality. Application architecture becomes less straightforward. The applications we are building today The post Data Intensive Applications with Martin Kleppmann appeared first on Software Engineering Daily.

We view our iPhones as inanimate objects. But when we see robots such as the Boston Dynamics machines that move with a motion that seems like an animal, the robot comes alive. We feel more sympathy and connection towards it. Today’s episode is about the distinction between inanimate machines and machines that seem alive. Peeqo The post Robot Assistant with Abhishek Singh appeared first on Software Engineering Daily.

Facebook’s open source projects include React, GraphQL, and Cassandra. These projects are key pieces of infrastructure used by thousands of developers–including engineers at Facebook itself. These projects are able to gain traction because Facebook takes time to decouple the projects from their internal infrastructure and clean up the code before releasing them into the wild. The post Facebook Open Source with Tom Occhino appeared first on Software Engineering Daily.

Engineers in Silicon Valley see a world of constant progress. Our work is creative and intellectually challenging. We are building the future and getting compensated quite well for it. But what if we are actually achieving far less than what is possible? What if, after so many years of high margins, gourmet lunch, and self-flattery, The post Complacency with Tyler Cowen appeared first on Software Engineering Daily.

Brendan Eich created the first version of JavaScript in 10 days. Since then JavaScript has evolved, and Brendan has watched the growth of the web give rise to new and unexpected use cases. Today Brendan Eich is still pushing the web forward across the technology stack with his involvement in the WebAssembly specification and the The post WebAssembly with Brendan Eich appeared first on Software Engineering Daily.