Software Engineering Daily

In this episode we discuss plug and play auth, password management, and crypto with Sean Li, co-founder and CEO of Magic. This interview was also recorded as a video podcast. Check out the video on the Software Daily YouTube channel. Sponsorship inquiries: [email protected]

For some data problems, you may be more concerned with the state of data at a particular point. A ticket is booked, or it’s not. How many poetry submissions were made to the contest? This is relational data. For other problems, you’re concerned with the change in data over time. Solar energy consumption, for example, or price behavior. This is time-series data. TimescaleDB resembles a traditional postgreSQL database, but is supercharged for time-series data. TimescaleDB has queries that are 10x faster, is optimized for time-series and advanced time-series analytics, has automated continuous aggregations, columnar storage, and uses best-in-class algorithms and memory efficient structures to compress your data so you can store more at a much cheaper price. And for your questions that are not time-series dependent, TimescaleDB is still an efficient and cost effective relational database. In this episode we talk to Mike Freedman, Co-Founder and CTO of TimescaleDB. Sponsorship inquiries: [email protected]

The cloud has delivered amazing benefits like on-demand infrastructure that’s easy to use, pay-as-you-go subscription plans, and effortless scaling of applications. This flexibility minimizes the growing pains for businesses and explains why today’s startups and established companies are both building apps on the cloud. However, the costs of using the cloud stack up once companies reach large scale. Dropbox, for example, shifted away from the cloud in 2016 to opt for a custom built infrastructure in co-location facilities. They saved $75 million over 2 years and increased their gross margins from 33% to 67% due to this change. Actually, recent research suggests major companies spend 75%-80% of cost of revenue on their cloud bills. Should large companies shift away from the cloud like Dropbox did? Is it too late for some companies to untangle themselves from it? In this episode we talk with Martin Casado, Partner at Andreessen Horowitz and an expert in IT Infrastructure. We discuss the costs of cloud for small and large companies, and the financial implications of cloud infrastructure at scale. Show NotesMartin Casado's article on cloud costs Sponsorship inquiries: [email protected]

Uber is one of many examples we’ve discussed on this show that has changed the world with big data analysis. With over 8 million users, 1 billion Uber trips and people driving for Uber in over 400 cities and 66 countries, Uber has redefined an entire industry in a very short time frame. It’s difficult to find precise details about Uber’s big data infrastructure online, but we know they collect every possible data point about their drivers and riders. Matching riders and drivers, setting ride fares, predicting demand for cars - these are some examples of what Uber does with its data. In this episode we talk with Kevin Novak about Uber’s data science. What are some key details about their data infrastructure? What can people expect in the future from their data methodologies? How did a tech conference in Paris turn into one of the fastest growing, highest valued startups in the world? Sponsorship inquiries: [email protected]

The quantity and quality of a company’s data can mean the difference between a major success or major failure. Companies like Google have used big data from its earliest days to steer their product suite in the direction consumers need. Other companies, like Apple, didn’t always use big data analytics to drive product design, but they do now. The company Axiom has created a large suite of advanced browser robots that perform difficult tasks like consolidating data across many web applications, extracting data from public sites or from behind logins, data entry, user interface automation, file management and spreadsheet automation. These powerful tools enable people and businesses to collect valuable data to inform their decisions. In this episode we talk to Yaseer Sheriff, Co-Founder and CEO at Axiom. We discuss the value of big data, the opportunities their products enable, and how people can use their tools to improve their data collection practices. Sponsorship inquiries: [email protected]

In this episode we discuss software investing, business, and the future with David Rosenthal, co-host of the Acquired podcast. This interview was also recorded as our very first video podcast. Check out the video on the Software Daily YouTube channel. Sponsorship inquiries: [email protected]

Complete information games are games where every player has information about the game sequence, strategies, and payoffs throughout gameplay. Playing chess, for example, relies on knowing the location of every piece everywhere on the board. In an incomplete information game like Minecraft, you continually gain new information during gameplay. Until very recently, incomplete information was nearly impossible on blockchains because every transaction and party is public. However, applied zero-knowledge cryptography on Ethereum has advanced greatly the past 18 months and is changing the transparency of blockchains. In this episode we talk to Brian Gu, a creator of the new game Dark Forest. Dark Forest is a space-conquest game built on Ethereum where players discover and conquer planets in an infinite, procedurally-generated, cryptographically-specified universe. It uses zero-knowledge proofs to validate game moves on Ethereum without sharing information to other players about the moves. We discuss the technology behind Dark Forest and the possibilities of a potentially less transparent blockchain future. Sponsorship inquiries: [email protected]

Proof of Work cryptocurrency mining, as used on the Ethereum and Bitcoin blockchains, requires huge amounts of energy to validate transactions and generate new tokens. The alternative, Proof of Stake, needs large deposits of assets to be staked up front in order to work. While both consensus protocols have their own drawbacks, they are the current industry standards. The company Spacemesh developed a new consensus protocol with the goal of powering an energy-efficient, decentralized, secure, and scalable smart contracts global computer and a cryptocurrency in the permissionless settings. They call their protocol Proof of Space Time which works on a blockmesh structure rather than a blockchain. The Spacemesh protocol allows newcomers to contribute to the security of the cryptocurrency network via unused storage space on their hard drives, driving down energy consumption and enabling anyone with a computer to contribute. In this episode we talk to Anton Learner, Core Team Lead at Spacemesh. Sponsorship inquiries: [email protected]

The company StreamSets is enabling DataOps practices in today’s enterprises. StreamSets is a data engineering platform designed to help engineers design, deploy, and operate smart data pipelines. StreamSets Data Collector is a codeless solution for designing pipelines, triggering CDC operations, and monitoring data in flight. StreamSets Transformer uses Apache Spark to generate insights about your data across multiple different platforms. Their Control Hub is the single hub for managing all of your data pipelines, data processing jobs, and execution engines. In this episode we talk to Arvind Prabhakar, CTO at StreamSets. Arvind is also an Official Member of the Forbes Technology Council, and a Member, PMC Chair/Member, Committer, Mentor, and Contributor to multiple projects with the Apache Software Foundation. He was previously a Director of Engineering at Cloudera, and a Software Architect at Informatica before that. Sponsorship inquiries: [email protected]

Delivering Saas products involves a lot more than just building the product. Saas management involves customer relationship management, licensing, renewals, maintaining software visibility, and the general management of the technology portfolio. The company Blissfully helps businesses manage their SaaS products from within a complete IT platform with organization, automation, and security built in. The Blissfully platform offers a system of record for creating and maintaining a single source of truth for technology, a workflows and automations feature for defining and executing consistent IT processes, an IT collaboration feature, and a security and compliance feature. These features come together to form a comprehensive IT management platform. In this episode we talk with Aaron White, a Founder and CTO at Blissfully. Aaron was previously a Co-Founder and Board Member of Price Intelligently, and Vice President at Venrock before that. Sponsorship inquiries: [email protected]

Amundsen was started at Lyft and is the leading open-source data catalog with the fastest-growing community and the most integrations. Amundsen enables you to search your entire organization by text search, see automated and curated metadata, share context with co workers, and learn from others by seeing most common queries on a table or frequently used data. Powered by Amundsen, the company Stemma is a fully managed data catalog that bridges the gap between data producers and data consumers. Stemma adds features to Amundsen like showing meaningful data to individual users, adding metadata to data automatically, and documenting data on the fly. Stemma integrates with all the major data sources like Snowflake, Redshift, Google BigQuery, and Apache Airflow. In this episode we talk to Mark Grover, Founder at Stemma. Mark co-created Amundsen and authored the book Hadoop Application Architectures. He was an engineer at Cloudera before joining Lyft as a Product Manager. Sponsorship inquiries: [email protected]

Coinbase is a very popular and well trusted cryptocurrency platform for buying and selling digital assets like Bitcoin, Ethereum, and many more. With Coinbase you can manage your portfolio of cryptocurrencies in 1 place like you would for other investments. There are added features like scheduling recurring purchases of assets, time-delayed withdrawals from digital vaults, and mobile apps with sleek UIs for mobile access to the markets. Coinbase has gained trust as a platform for storing digital assets in secure offline storage, using servers covered by insurance policies, by following industry best practices and supporting a variety of the most popular digital currencies. In this episode we talk to Brent Walter, a Senior Engineering Manager at Coinbase. Brent was previously an Advisor at With Labs and various other companies, and was a Director of Software Engineering at Western Digital before that. Sponsorship inquiries: [email protected]

Corey Quinn is guest hosting on Software Engineering Daily this week, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. In this episode, Corey Quinn interviews Salman Paracha, Group Vice President, Cloud Engineering at Oracle.

Corey Quinn is guest hosting on Software Engineering Daily this week, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. In this episode, Corey Quinn interviews John Allspaw, Founder and Principal of Adaptive Capacity Labs.

Corey Quinn is guest hosting on Software Engineering Daily this week, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. In this episode, Corey Quinn interviews Liz Fong-Jones, who is Principal Developer Advocate for SRE and Observability at Honeycomb and who worked at Google prior to that.

Corey Quinn is guest hosting on Software Engineering Daily this week, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. In this episode, Corey Quinn interviews Troy Hunt, who is a Microsoft Regional Director and MVP. Troy also runs "Have I Been Pwned" which checks to see if you have an email or password that has been compromised in a data breach.

Corey Quinn is guest hosting on Software Engineering Daily this week, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. In this episode, Corey Quinn interviews Pete Cheslock, Chief Product Officer at Allma, about AWS. Prior to that Pete was Corey's colleague at The Duckbill Group.

Mark Saroufim is the author of an article entitled “Machine Learning: The Great Stagnation”. Mark is a PyTorch Partner Engineer with Facebook AI. He has spent his entire career developing machine learning and artificial intelligence products. Before joining Facebook to do PyTorch engineering with external partners, Mark was a Machine Learning Engineer at Graphcore. Before that he founded Yuri.ai. Mark has also published “The Robot Overlord Manual” which “will teach you all the software, math and ML you’ll need to start building robots at home.” In this episode we discuss machine learning subjects and his experience developing cutting edge software. Sponsorship inquiries: [email protected]

Next week Corey Quinn will be guest hosting on Software Engineering Daily, presenting a Tour of the Cloud. Corey Quinn is the Chief Cloud Economist at The Duckbill Group, where he helps companies fix their AWS bill by making it smaller and less horrifying. If you're looking to lower your AWS bill or negotiate a new contract with AWS, you can learn more about The Duckbill Group's services at https://www.duckbillgroup.com/. Corey is also the host and creator of Last Week in AWS, which publishes newsletters and podcasts covering topics to help you stay up to date on all things AWS and insightful conversations with experts in the world of cloud computing all delivered lovingly with Corey's infamous snark. Subscribe at https://www.lastweekinaws.com and follow Corey on Twitter @QuinnyPig. Amazon Web Services changed how software engineers work. Before AWS, it was common for startups to purchase their own physical servers. AWS made server resources as accessible as an API request, and has gone on to create higher-level abstractions for building applications. For the first few years of AWS, the abstractions were familiar. S3 provided distributed, reliable object storage. Elastic MapReduce provided a managed Hadoop system. Kinesis provided a scalable queue. Amazon provided developers with managed alternatives to complicated open source software. More recently, AWS has started to release products that are unlike anything else. A perfect example is AWS Lambda, the first function-as-a-service platform. Other newer AWS products include Ground Station, a service for processing satellite data and AWS DeepRacer, a miniature race car for developers to build and test machine learning algorithms on. As AWS has grown into new categories, the blog announcements of new services and features have started coming so frequently that it is hard to keep track of it all. Corey Quinn is the author of “Last Week in AWS”, a popular newsletter about what is changing across Amazon Web Services. Corey joins the show to give his perspective on the growing, shifting behemoth that is Amazon Web Services--as well as the other major cloud providers that have risen to prominence. He’s also the host of the Screaming in the Cloud podcast, which you should check out if you like this episode.

Platforms like Ethereum have billions of dollars of market cap and large developer communities. However, it is still a challenge to build widely adopted DApps on it because of current limitations. Blockchain Proof of Work transactions are typically slow, and Proof of Stake transactions trade off decentralization to achieve high throughput. Transaction fees get expensive, especially for high network load times, scalability is low and this creates a bad user experience. The company Polygon (previously Matic Network) solves some of these problems with their platform for Ethereum scaling and infrastructure development. Polygon combines the features of stand-alone blockchains (like sovereignty, scalability, and flexibility) and Ethereum (security, interoperability and developer experience). These features enable scalable solutions on Ethereum and support a multi-chain Ethereum ecosystem (polygon.technology). In this episode we talk to Denis Ermolin, a Senior Software Engineer at Polygon. Denis was previously a Senior Software Engineer at Animoca Brands, and CEO of Moonrealm Entertainment before that. Sponsorship inquiries: [email protected]

AWS Outposts is a fully managed service that offers the same AWS infrastructure, AWS services, APIs, and tools to virtually any datacenter, co-location space, or on-premises facility for a truly consistent hybrid experience. AWS Outposts is ideal for workloads that require low latency access to on-premises systems, local data processing, data residency, and migration of applications with local system interdependencies (aws.amazon.com). In this episode we talk with Joshua Burgin, General Manager, AWS Outposts at Amazon Web Services. Joshua owns strategy, roadmaps, customer experience, pricing and demand generation for the AWS Edge/Hybrid Compute business, including full P&L responsibility. Joshua was previously a Senior Director, Technology Platform and Services at Zynga and a Senior Manager, Product Management at RPI before that. Sponsorship inquiries: [email protected]

Flutter is a UI toolkit developed by Google that helps developers build natively compiled applications for mobile, web, desktop, and embedded devices from a single code base. Development is fast because the screen “hot reloads” as you develop, the architecture is layered for fast and expressive designs, and its widgets incorporate all critical platform differences such as scrolling, navigation, icons and fonts. In this episode we talk about developing Flutter apps with Allen Wyma. Allen is a Founder of Plangora, a web and mobile development company that specializes in PHP, iOS with Swift, android with Java, Wordpress, and Flutter. Allen is also an Elixir Mix Panelist at Devchat.tv where he interviews members of the Elixir community. We discuss Native app development and developing with Flutter. Sponsorship inquiries: [email protected]

Data exploration uses visual exploration to understand what is in a dataset and the characteristics of the data. Data scientists explore data to understand things like customer behavior and resource utilization. Some common programming languages used for data exploration are Python, R, and Matlab. Doris Jung-Lin Lee is currently a Graduate Research Assistant at the University of California, Berkeley, also earning a PhD in Information Management and Systems. Doris also did her undergrad at Berkeley, studying physics and astrophysics. She is currently developing Lux, a Python library for accelerating and simplifying the process of data exploration. Her research and work with Lux is aimed to make data science more intuitive and accessible to end users. In this episode Doris joins us to discuss data exploration and her research and development of Lux. Sponsorship inquiries: [email protected]

Arun Kumar is an Assistant Professor in the Department of Computer Science and Engineering and the Halicioglu Data Science Institute at the University of California, San Diego. His primary research interests are in data management and systems for machine learning/artificial intelligence-based data analytics. Systems and ideas based on his research have been released as part of the Apache MADlib open-source library, shipped as part of products from Cloudera, IBM, Oracle, and Pivotal, and used internally by Facebook, Google, LogicBlox, Microsoft, and other companies. Arun did his undergrad in Computer Science and Engineering at the Indian Institute of Technology, Madras, and then his MS and PhD in Computer Science at the University of Wisconsin-Madison, where his thesis research explores problems at the intersection of data management and machine learning, with a focus on problems related to usability, developability, performance, and scalability. In this episode he joins us to discuss data management systems and artificial intelligence. Sponsorship inquiries: [email protected]

Cloud data warehouses are databases hosted in cloud environments. They provide typical benefits of the cloud like flexible data access, scalability, and performance. The company Firebolt provides a cloud data warehouse built for modern data environments. It decouples storage and compute to operate on top of existing data lakes like S3. It computes orders of magnitude faster performance from gigabyte to petabyte scale by using a columnar data structure, vectorized processing, just-in-time query compilation, and continuously aggregated indexing. Firebolt scales with data lakes by processing queries across clusters of nodes in parallel, providing consistently fast processing and granular control over resources. In this episode we talk with Eldad Farkash, Co-Founder and CEO of Firebolt. Eldad was previously a Venture Partner at Angular Ventures and a Founder, CTO and Board Member at Sisense before that. We discuss big data, data warehouses, and the unique benefits offered by Firebolt. Sponsorship inquiries: [email protected]

Running applications in containerized environments involves regularly organizing, adding and replacing containers. This complex job may involve managing clusters of containers in different geographic locations with different configuration requirements. Platforms like Kubernetes are great for managing this complexity, but include steep learning curves to efficiently get anything off the ground. The company Portainer provides a universal container management tool that works with Kubernetes, Docker, Docker Swarm, and Azure ACI. It enables managing containers without knowing platform-specific code and best practices. Instead, deploying containerized applications is done through a simple Graphical User Interface. Once deployed, you can observe and monitor the apps and govern security settings all through Portainer. In this episode, we talk to Neil Cresswell, a Co-Founder at Portainer. Neil is also CEO and Founder of CloudInovasi, and was CEO at Emerging Technology Partners before that. Sponsorship inquiries: [email protected]

Apache Superset is an open-source, fast, lightweight and modern data exploration and visualization platform. It can connect to any SQL based data source through SQLAlchemy at petabyte scale. Its architecture is highly scalable and it ships with a wide array of visualizations. The company Preset provides a powerful, easy to use data exploration and visualization platform powered by Apache Superset. Preset enables team members with some to no programming experience to build interactive visualizations and dashboards with a no-code viz builder and SQL editor. It works directly on top of popular cloud data warehouses and leading data engines. Preset delivers all the data visualization power of Apache Superset through their complete, easy to consume, enterprise ready platform. In this episode we talk with Srini Kadamati, Senior Data Scientist / Developer advocate at Preset. Previously Srini worked as Head of Product at Dataquest.io and as a Data Scientist at Radius Intelligence before that. He is also a Committer to Apache Superset. We discuss data visualization, the power of big Data, and Preset. Sponsorship inquiries: [email protected]

Application Programming Interfaces (APIs) are interfaces that enable multiple software applications to send and retrieve data from one another. They are commonly used for retrieving, saving, editing, or deleting data from databases, transmitting data between apps, and embedding third-party services into apps. The company BaseTen helps companies build and deploy machine learning APIs and applications. Using pre-existing ML models, or choosing from BaseTen’s library of pretrained models, BaseTen helps you instantly deploy API endpoints powered by those models to use in your applications. These APIs easily scale and integrate with existing data sources. BaseTen’s serverless infrastructure enables chaining model outputs and pre- and post- processing code. They also use a drag-and-drop UI builder to create custom UI’s for the applications, all without learning React. In this episode, we talk with Tuhin Srivastava and Amir Haghighat, founders at BaseTen. Tuhin previously founded Shape, and also worked as a Data Scientist at Gumroad. We discuss machine learning API development, scaling ML-driven applications, and the capabilities of BaseTen’s technology. Sponsorship inquiries: [email protected]

The company Skynet Labs provides an open protocol for hosting data and web applications on the decentralized web. Skynet allows for decentralized, censorship-resistant, highly redundant storage and applications that are available around the globe. Developers don’t pay for their application’s storage, can launch apps with access to a user’s data right away, are free from corporations pulling access to their resources, and can maintain failover sites for when their primary site goes down. For users, you take your data with you without any corporate oversight, support developers and content creators by simply accessing their work, experience a web free of targeted ads, and never have to put your privacy or security at risk. Skynet is built on top of the Sia blockchain network, which is open-source and guided by the Sia Foundation. In this episode we talk with Matthew Sevey, Engineering Engineer at Skynet Labs. Before joining Skynet, Matthew was a Project Manager at Procter and Gamble and a web development fellow at Startup Institute before that. We discuss decentralized internet, the Sia blockchain network, and Skynet’s mission to build a better and more equitable ecosystem. Sponsorship inquiries: [email protected] Show Notes:Skynet Website Skynet Jobs Skynet Discord Skynet Hackathon SkyChat SkyFeed

Columnar databases store and retrieve columns of data rather than rows of data. Each block of data in a columnar database stores up to 3 times as many records as row-based storage. This means you can read data with a third of the power needed in row-based data, among other advantages. The company Altinity is the leading enterprise provider for ClickHouse - an open-source column-store analytic database, now a fully managed service developed and operated with Altinity.Cloud. Altinity only bills for the compute, storage, and support that is used. They provide enterprise support for analytic applications like tuning queries, Kafka support, and ClickHouse bugs, and their ClickHouse clusters run with out-of-the-box security and privacy. In this episode we talk with Robert Hodges, CEO at Altinity. Before becoming CEO at Altinity, Robert worked as a Senior Staff Engineer at VMWare and was the CEO of Continuent before that. We discuss databases and data warehousing, ClickHouse, and how Altinity helps customers create enterprise analytic applications. Sponsorship inquiries: [email protected]

Apache Spark is a popular open source analytics engine for large-scale data processing. Applications can be written in Java, Scala, Python, R, and SQL. These applications have flexible options to run on like Kubernetes or in the cloud. The company Data Mechanics is a cloud-native Spark platform for data engineers. It runs continuously optimized Apache Spark workloads on a managed Kubernetes cluster within the user’s cloud account. They boast a 50%-75% cost reduction from cloud providers by dynamically scaling applications based on load and automatically tuning app configurations based on the historical Spark pipeline runs. Their Kubernetes clusters are deployed within user accounts so user data never leaves the environment and they handle the cluster management. In this episode we talk to Jean-Yves Stephan, Co-Founder and CEO at Data Mechanics. Jean-Yves previously worked as a Software Engineer then a Tech Lead Manager at Databricks. We discuss big data engineering in Spark and the unique advantages of using Data Mechanics to make Spark development easier and more cost effective. Sponsorship inquiries: [email protected]

Apache Hudi is an open-source data management framework used to simplify incremental data processing and data pipeline development. This framework more efficiently manages business requirements like data lifecycle and improves data quality. Some common use cases for Hudi is record-level insert, update, and delete, simplified file management and near real-time data access, and simplified CDC data pipeline development (AWS.amazon.com). In this episode we speak to Vinoth Chandar, VP of Apache Hudi. Vinoth is the creator of the Hudi project at Uber. He continues to lead its evolution at the Apache Software Foundation. Previously he was a Principal Engineer at Confluent, and a Sr Staff Engineer/Manager at Uber before that. We discuss building large scale distributed and data systems. Sponsorship inquiries: [email protected]

An application programming interface, API for short, is the connector between 2 applications. For example, a user interface that needs user data will call an endpoint, like a special URL, with request parameters and receive the data back if the request is valid. Modern applications rely on APIs to send data back and forth to each other and save, edit, delete, or retrieve data in databases. The number of APIs used in a single application is growing due to the increase of micro-services and distributed architectures. Understanding how your applications use APIs can increase their efficiency and stability and make debugging easier. The company Akita observes the structure of programs to visualize, map, and manage API behavior. By monitoring the APIs in your applications, Akita can catch code changes that may break production applications. While this work is normally labor-intensive, Akita automates it by analyzing the source code and logs. They check the observed behaviors against intended specs and contracts to provide clear oversight on all activity. This information can then be generated into maps that help you document and version your APIs across your entire service ecosystem. In this episode we talk with Jean Yang, Founder and CEO of Akita Software. Jean was previously an assistant professor at Carnegie Mellon University and a postdoctoral researcher and Harvard Medical School before that. We discuss modern APIs, their role in applications, and how Akita Software makes understanding and building APIs easier for developers. Sponsorship inquiries: [email protected]

The traveling salesman problem is a classic challenge of finding the shortest and most efficient route for a person to take given a list of destinations. This is one of many real-world optimization problems that companies encounter. How should they schedule product distribution, or promote product bundles, or define sales territories? The answers to these questions constantly change because business environments constantly change. The company Nextmv helps solve these problems with production-ready, commercial tools for solving optimization problems and simulating models with real company data. Their tool Hop encodes optimization strategies for dynamic environments. Hope can be deployed to routing, scheduling and assignment problems in multiple industries like on-demand delivery, e-commerce, and IT infrastructure management. Their tool Dash is a commercial-grade simulation engine that provides an environment to “A/B test” models online with real data. In this episode we talk to Carolyn Mooney, CEO at Nextmv. Carolyn was previously a Lead Systems Engineer at Grubhub, and a Decision System Analyst at Zoomer before that. We discuss optimization problems throughout different industries, machine learning strategies for solving them, and go into detail about how Nextmv helps companies become more profitable and efficient. Sponsorship inquiries: [email protected]

Microservice architecture has become very common over the past few years because of the availability of containers and container orchestrators like Kubernetes. While containers are overall positive for scaling apps and making them more available, they’ve also introduced hurdles like persisting data and state, and container restarts or pod failures. Development teams put significant work into designing applications that take these hurdles into account because without precautions you can lose valuable data or crash your app. The company Temporal provides tools for both building complex microservices as well as for apps that use microservices. They use 2 primary function types: workflow and activity. Workflow functions persist all local variables and threads so that if the server the app runs on crashes, it’s picked up on a different server where it left off, down to the line. Activity functions automatically initiate retry logic if the service the function invokes fails for something like its server being down. Temporal provides visibility into end-to-end workflows that can span multiple services. In this episode we talk to Ryland Goldstein, Head of Product at Temporal. Previously, Ryland was the Lead Product Manager at Reshuffle and a Software Engineer Lead at Parallel Machines. We discuss the challenges of managing state in micro services, orchestrating microservices and how Temporal simplifies this process for development teams. Sponsorship inquiries: [email protected]

Natural Language Processing (NLP) is a branch of artificial intelligence concerned with giving computers the ability to understand text and spoken words. “Understanding” includes intent, sentiment, and what’s important in the message. NLP powers things like voice-operated software, digital assistants, customer service chat bots, and many other academic, consumer and enterprise tools. The company Botpress provides open-source developer tools to create NLP tools for process and FAQ automation. They use the latest NLP models for domain-specific, contextual and goal-oriented conversations. This technology is free and available through simple API routes. They also maintain integrations with popular messaging services like Facebook Messenger, Slack, and Microsoft Teams. For other proprietary systems, they provide a raw Messaging API. In this episode we talk to Sylvain Perron, CEO of Botpress. Sylvain was previously a Director of Engineering at Protorisk Limited and a Software Developer at ArcBees before that. We discuss the current advances in Natural Language Processing and how NLP powers Botpress. Sponsorship inquiries: [email protected]

In software engineering, telemetry is the data that is collected about your applications. Unlike logging, which is used in the development of apps to pinpoint errors and code flows, telemetry data includes all operational data including logs, metrics, events, traces, usage, and other analytical data. Companies usually visualize this information to troubleshoot problems and understand patterns and opportunities in how their applications are used. The company New Relic is a modern observability platform built to optimize your entire software stack from one place. New Relic includes a Telemetry Data Platform that acts as a single source of truth for telemetry data. Built on top of that are tools for full stack observability to visualize and troubleshoot your data in milliseconds. When a problem does occur, New Relic’s applied intelligence will detect and understand the problem to help resolve it faster. In this episode we talk with Zain Asgar and Ishan Mukherjee. Zain is currently the GVP/GM - Pixie at New Relic, through the acquisition of Pixie Labs Inc where he was the co-founder and CEO. Ishan leads GTM for all New Relic products and is a co-founder and CPO of Pixie. We discuss big data, application observability, and how New Relic provides data observability and intelligence in one platform. Sponsorship inquiries: [email protected]

Cloud computing provides tools, storage, servers, and software products through the internet. Securing these resources is a constant process for companies deploying new code to their cloud environments. It’s easy to overlook security flaws because company applications are very complex and many people work together to develop them. Wyze Labs, for example, had millions of users’ data stolen due to a mistake by a single employee. The company Bridgecrew is a cloud security platform helping to prevent mistakes like that from happening. Bridgecrew integrates into developer workloads to automatically find infrastructure errors in cloud accounts, workloads, and infrastructure as code. Their platform also monitors code reviews and build pipelines to prevent errors from being deployed into production. If an error is found then Bridgecrew’s software reverts that code back to its last known correct state. In today's episode we talk with Barak Schoster, CTO and co-founder at Bridgecrew. Barak previously worked as a senior software architect at RSA Security and as a software architect at Fortscale before that. We discuss cloud security, Infrastructure as Code, and big data architecture. Sponsorship inquiries: [email protected]

Modern applications are increasingly built as large, distributed systems. A distributed system is a program where its components are located on different machines that communicate with one another to create a single cohesive app. Components may exist as multiple instances across “nodes,” the computers hosting them, which form clusters of nodes that span across geographic regions. The platform Uber is built on distributed systems. The benefit of this architecture is higher availability, higher load capacity, and lower latency. This is essential for Uber because it needs to process up to thousands of requests per second. But this architecture also presents challenges. Some data, like payment requests, needs to be 100% accurate, which can be challenging for a system where its data is spread out across multiple nodes. The data needs to be durable as well-- if a node fails, the data it holds needs to be readily available on another. Their nodes also need to be up-to-date with the information from other nodes so they process the right requests. Any company using distributed systems faces these challenges. In this episode we talk with Gergely Orosz. Gergely is a software engineer and author of The Software Engineer’s Guidebook and his blog The Pragmatic Engineer. He previously worked as an engineering manager at Uber, working on their payments experience platform. We discuss distributed systems, payment technology, and the challenges he overcame working at Uber. Sponsorship inquiries: [email protected]

Geospatial technology impacts every person who uses a smartphone, drives a car, or flies in airplanes. It refers to all of the technology used to acquire and interpret geographic information. In more advanced settings, geospatial technology is used for constructing dynamic maps, 3D visualizations, and scientific and governmental simulations. The company Makepath specializes in geospatial technology and full-stack application development. Makepath helps companies to create beautiful and simple visualizations from mountains of complex data. Using open-source Python libraries and real-world validation, they create state-of-the-art analytics, web applications, and other robust automation processes. They are also passionate about the open source ecosystem and contribute to many ongoing projects. In this episode we talk with Brendan Collins from Makepath. Brendan is a founder and the principal of Makepath. He previously worked as the principal of Parietal and was a software developer at Anaconda, Inc before that. We discuss geospatial data science, the work they do at Makepath, and their broader open source projects. Sponsorship inquiries: [email protected]

Large portions of software development budgets are dedicated for testing code. A new component may take weeks to thoroughly test, and even then mistakes happen. If you consider software defects as security issues then the concern goes well beyond an application temporarily crashing. Although even minor bugs can cost companies a lot of time to locate the bug, resolve it, retest it in lower environments, then deploy it back to production. The company Speedscale provides an intelligent, Kubernetes-friendly testing toolkit that runs at build time. Their virtual SRE-bot works inside automated release pipelines to forecast and test real-world conditions the new code will encounter. This process requires no manual scripting because Speedscale uses existing traffic to generate tests and mocks. The feedback is immediate after every build and covers regression, performance, fuzzing, and chaos tests automatically. In this episode we talk with Ken Ahrens and Matt LeRay. Ken is a founder and the CEO at Speedscale. Previously Ken worked at New Relic as a senior director, solutions architects. Matt is a founder and CTO at Speedscale. He previously was the VP of Product at Observe, inc. We discuss testing in distributed environments, how Speedscale intelligently tests and mocks during builds, Kubernetes, and their future goals with Speedscale. Sponsorship inquiries: [email protected]

Encryption algorithms provide the means to secure and transfer sensitive information by taking input and transforming it into an unreadable output. Usually a special key, or multiple keys, are needed to unscramble the information back to the original input. These algorithms power the security of everything from our cell phone lock screens to Fortune 500 company servers. The company Skiff is protecting data privacy with their first product, the only end-to-end encrypted document collaboration platform with password protected folders, expiring links, and secure workplaces. Skiff’s document platform has all the traditional features of a typical document editor, making it feel familiar and comfortable. Their end-to-end encryption and built-in password protection are 2 of several methods that make collaborating on documents more safe and within user control than on any other platform. In this episode we talk with Andrew Milich, CEO of Skiff. Andrew was previously an associate product manager at Schmidt Futures. We discuss data privacy and security, the Skiff document collaboration platform, and potential future security products. Sponsorship inquiries: [email protected]

A liquid market enables individuals or groups to quickly buy and sell assets. Decentralized platforms can struggle to execute trades when their platform does not have much liquidity for a specific token. Newer tokens or tokens with limited supply are most often the least liquid because there might be an imbalance of buyers and sellers. You can’t sell token A for price X without a consenting buyer on the other end. The company Uniswap is a decentralized protocol for creating liquidity and trading ERC-20 tokens on Ethereum. Uniswap encourages users to be “liquidity providers” whereby they pool their assets into funds that enable people to complete trades without an opposite party. Instead, they swap against the liquidity pool created by the liquidity providers. Every swap incurs a small fee, which is distributed proportionately to liquidity providers when they decide to pull their funds. Uniswap prices coins based on the simple formula x*y=k. In this episode we talk with Noah Zinsmeister, engineering lead at Uniswap. Noah also maintains web3-react, a framework for building blockchain applications. We discuss cryptocurrency liquidity, the Ethereum blockchain, and how Uniswap is building a community of liquidity providers and traders. Sponsorship inquiries: [email protected]

Non-fungible tokens are proofs of authenticity that are stored on a blockchain. Unlike fungible tokens, such as cryptocurrencies which are interchangeable, non-fungible tokens aren’t inherently equivalent to any other token. Because they are unique, they can be used to represent any unique asset. Their presence on a blockchain enables an NFT owner to trade the asset and prove they are its owner. One such use case for NFTs is artwork, and the company Showtime is bringing artists and creators together to showcase their work. Crypto art is verified with NFTs and therefore can’t be authentically reproduced. This lets artists sell their original work to prospective buyers. With Showtime, you can discover and follow new crypto artists, browse the trending page and like, comment, and share artwork. The platform shows a variety of art styles and experience levels. In this episode we talk with Alex Masmejean, co-founder and CEO at Showtime. Before Showtime Alex worked as an advisor at TapIn and operations at MetaCartel DAO. We discuss non-fungible tokens, the world of crypto art and the growth and future of Showtime. Sponsorship inquiries: [email protected]

Volatility is the degree of fluctuation of something’s price. Highly volatile assets may see rapid and large price changes, while less volatile assets will maintain a steady price. This concept is important in decentralized finance because cryptocurrencies tend to be volatile assets. The company Synthetix provides assets called Synths that provide exposure to an asset without holding the underlying resource. For example, you can hold and trade Synths that track the price of USDs, synthetic gold and silver (measured by the ounce), and other currencies and commodities. Users use Synthetix to trade the price equivalents of real-world assets on Ethereum. This lets them diversify their investment portfolios with less volatile assets while staying on the blockchain and executing trades against smart contracts. In today’s episode we talk with Justin Moses, CTO at Synthetix. Previously Justin worked as CTO at Haven and as a Tech Advisor at blueshyft. We discuss derivatives trading in DeFi, the liquidity and volatility of synthetic assets, and the rewards and features available from using Synthetix. Sponsorship inquiries: [email protected]

A decentralized exchange, usually referred to as a DEX, is a platform for exchanging cryptocurrencies. Depending on trading volume for different coins, some DEXs are more liquid than others. On the one hand you can freely swap unlisted tokens and maintain full control over your private keys and wallet information. On the other hand, without the right supply and demand it’ll be difficult to swap a particular coin at the right price. 1inch Exchange is a decentralized exchange aggregator that can split a single trade transaction across multiple DEXs. Their Pathfinder API ensures users get the best price by using a discovery and routing algorithm to find the best possible paths for token swaps and then splits the swap across multiple exchanges and market depths of the same exchange. 1inch uses multiple liquidity sources, including private liquidity providers, to ensure there is sufficient liquidity for all swaps on their platform. If the rate of a trade becomes more expensive than the user has confirmed from the UI, the algorithm can cancel part of the route and simply return the unswapped tokens to the user’s wallet. In today’s episode we talk with Anton Bukov, co-founder of 1inch. Anton was previously a senior smart contract engineer at NEAR Protocol and chief blockchain engineer, consensus researcher at MultiToken. We discuss the pros and cons of decentralized exchanges, splitting cryptocurrency swaps across multiple exchanges with 1inch Exchange, and the growth of decentralized finance. Sponsorship inquiries: [email protected]

A data warehouse is a data management system that often contains large amounts of historical data and is used for business intelligence activities like analytics. It centralizes customer data from multiple sources to be an organization’s single source of truth. Getting the data from your data warehouse into the different applications used by your organization can be difficult. The company Census simplifies syncing your data warehouse with other applications. Census works on top of existing infrastructure and lets users pick destinations apps, map the data, and then handles maintaining live, in sync metrics. Because Census runs inside your data warehouse, data remains secure and is never stored on their servers. In today’s episode we speak with Boris Jabes, CEO of Census. Boris was previously a managing partner at Polynome and a Senior Director at LogMeln. We discuss the complexity of data warehouses, how the field of data analytics has grown over the years, and how Census makes data accessibility easy and secure. Sponsorship inquiries: [email protected]

IT infrastructure are the components required to operate IT environments, like networks, virtual machines or containers, an operating system, hardware, data storage, etc…. As companies build out different deployment environments with infrastructure configurations, they must maintain the different environments, replicate them, and update them. The management of infrastructure, often automated to some extent, is referred to as Infrastructure as Code (IaC). The company Gold Fig Labs helps growing companies better understand their deployed infrastructure beyond the basic IaC principles. Gold Fig Labs developed 2 main tools to deliver the clearest view of infrastructure security and compliance. Their tool Checkup provides periodic security and best practices reports for AWS accounts. The report details specific, actionable, and relevant advice to improve security posture. Their other tool, Introspector, is a unique security and auditing tool that provides in-depth analysis of larger cloud deployments with complex regulatory requirements and custom internal policies. In this episode we talk with Vikrum Nijjar and Greg Soltis. Vikrum is co-founder and CEO at Gold Fig Labs. He was previously an angel investor with Angel and special advisor for Google’s onboarding acquisitions, compliance, and security. Greg is co-founder and CTO at Gold Fig Labs. Previously he was a senior software engineer at Google and a software engineer at Firebase. We discuss the principles of IaC, how Gold Fig Labs helps customers go beyond what IaC intends, and the complexity of cloud infrastructure security and regulatory compliance. Sponsorship inquiries: [email protected]

Decentralized applications, termed “dApps,” are applications that feel like normal apps but are actually deployed (mostly) on the Ethereum blockchain. This means dApps can’t be taken down, can’t be censored or blocked, typically use Ethereum accounts as identity, and would only experience downtime if Ethereum itself went down. There are a lot of things you can do with blockchain applications, particularly with decentralized finance. The company Compound develops protocols, built on the Ethereum blockchain, that establishes money markets. Money markets are pools of assets with algorithmically derived interest rates based on supply and demand. The Compound protocol represents assets as fungible ERC-20 token balances called cTokens. cTokens automatically increase in value from the amount of the initial underlying asset. The interest generated and managed through the Compound protocol can be used primarily for long-term investing in Ether and tokens as well as dApps and other entities. Compound provides lots of documents and discords for infusing interest and liquidity into dApps and related projects. This enables dApps to manage assets that generate interest and could lead to entirely new blockchain-based business models. In this episode we talk with Jared Flatow, Director of Protocol at Compound. Previously, Jared worked as a software engineer at Caffeine and founded the company Quasi Convex Union. We discuss the importance of liquidity and interest-earning assets in DeFi, how Compound is helping enhance dApps and the role and growth of dApps overall, and his goals for Compound going forward. Sponsorship inquiries: [email protected]

A ‘token’ can represent almost anything in Ethereum, according to Ethereum.org: Lottery tickets, points in an online platform, fiat currency, and much more. These tokens must follow a standard called ECR-20 to have the same type and value of any other token, and behave just like the ETH. The platform Opyn lets users buy and trade decentralized finance (DeFi) options on ETH and ECR20s. “Options” represent underlying assets with predefined (strike) prices and expiry dates that can be bought and sold. Opyn provides options protocols through smart contracts that are powerful and capital efficient. Using options helps secure against volatility and flash crashes, typically have lower margin requirements, can yield interest bearing collateral, and are noncustodial with Opyn. In this episode we speak with Aparna Krishnan, a co-founder of Opyn. We discuss options trading and how it differs from trading pure cryptocurrencies, developing and using smart contracts to define DeFi protocols, and the unique benefits of using Opyn for options trading. Sponsorship inquiries: [email protected]