The InfoQ Podcast

In this podcast, Marty Abbott and Tanya Cordrey sat down with InfoQ podcast co-host Daniel Bryant. Abbott, CEO and co-founder of AKF Partners, and Cordrey, partner at AKF Partners, discussed topics that included: their learning from working together in the early days of eBay, why and how to avoid creating software systems that are composed of deep call chains of microservices, and how to build effective product teams. Why listen to this podcast: - First introduced in the book “The Art of Scalability”, the AKF Scale Cube is a model for segmenting software components, defining microservices, and scaling products. It also creates a common language for teams to discuss scale related options in designing solutions. - The microservice architectural pattern is best used for implementing the “breadth” of business functionality. Engineers should avoid building deep call chains of services, as this can increase the probability of failure, and can also increase the challenges of locating and diagnosing issues. Code libraries can often be used more effectively to implement “depth” within services. - The AKF Availability Cube is a new model to guide discussions on how to achieve high availability. This model can also be used as a mathematical tool to evaluate the theoretical “as designed” availability of existing systems. - Building products using cross-functional teams is an effective approach. However, care should be taken not to accidentally create unnecessarily large teams, as this can add communication and coordination friction to the delivery process. - Teams should make a conscious choice to adopt new technologies, and understand the benefits and tradeoffs with doing so. Managing risk, and in particular, technology lifespan risk, is an important part of the value engineers provide to the business. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2AbpZjT You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2AbpZjT

In this podcast, Daniel Bryant sat down with Dave Sudia, senior DevOps engineer at GoSpotCheck. Topics discussed included: the benefits of PaaS; building a platform with Kubernetes as the foundation; selecting open source components and open standards in order to facilitate the evolution of a platform; and why care should be taken to prioritize the developer experience and create self-service operation of the platform. Why listen to this podcast: - When starting a business and searching for product-market fit, creating an application using a monolithic code base deployed onto a commercial Platform as a Service (PaaS) product is a very effective way of iterating fast and minimising operational costs. - There may come a point where the PaaS cannot provide bespoke requirements, or it has trouble scaling, or the costs become prohibitive. At this point many teams choose to build a custom platform using cloud technologies, such as Kubernetes. - Building a Kubernetes platform can be an effective solution, but appropriate effort needs to be put into designing, building, and maintaining the platform. The platform effectively becomes another product within the business that must be managed accordingly. - Embracing open standards provides many benefits, especially for the long term. Implementations that are consumed through well-defined interfaces and abstraction can be more readily swapped at a later point in time. It is also generally easier to integrate components that share common interfaces. - Attention and resources must be provided to create an effective developer experience for the platform. It is essential to prioritize self-service operations, and also to understand the core requirements of the engineers and QA specialists that will be using the platform during their daily work. - Establishing an effective continuous delivery pipeline can enable more repeatable and scalable testing of applications, and also allows the codification of cross-functional requirements. - The cloud native landscape has now evolved to a point where most of the frameworks and tooling required to build a platform have become viable for general purpose usage. However, some assembly may still be required, and engineers should be prepared for change, as the ecosystem moves fast. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bj3BTp You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3bj3BTp

Today on The InfoQ Podcast, Wes Reisz talks with Peter Bourgon. Peter is a distributed system engineer working on Fastly. His area of interest is around coordination free replicated systems. The two engineers talk about the space of Conflict-Free Replicated Data Types (CRDTs) specifically in the context of edge compute. Topics covered on the podcast include Edge compute, CRDTs, CAP Theorem, and challenges around building distributed systems. Why listen to this podcast: - CRDTs (conflict-free replicated data types) are a class of coordination free replication systems (or systems that don’t require strategies such as leader election). - An easy way to think of a CRDT is as an associative, commutative, and idempotent data structure plus the operations needed to use it. The edge is an overloaded term that people tend to define based on where they sit on a spectrum between the customer and the data center. Fastly’s edge is away from the data center and but not to the telephone pole or handset. - RAFT and Gossip are two alternative approaches to using a coordination free replication system like CRDTs. To get the properties of a CRDT and have useful data types, you have to pay a cost in size and often bytes on the wire. These are challenges that continue to need solutions. - Modern Distributed systems and data structures like CRDTs require you to start thinking about state in the system itself. It’s not unusual for a system today to give you multiple results back that the system will have to handle or merge. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dTZOO4 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3dTZOO4

In this podcast, Daniel Bryant sat down with Joe Duffy, founder and CEO at Pulumi, and discussed several infrastructure-themed topics: the evolution of infrastructure as code (IaC), the way in which the open source Pulumi framework allows engineers to write IaC using general purpose programming languages such as JavaScript and Go, and the future of multi-cloud environments. Why listen to this podcast: ● Infrastructure as Code (IaC) enables engineers to programmatically define the configuration and provisioning of computing infrastructure, on-premises hardware, and cloud services. ● Traditional IaC tools were often imperative, requiring engineers to define and enumerate the necessary steps and SDK calls in order to configure the underlying infrastructure. ● Modern IaC tools like HashiCorp’s Terraform, AWS CloudFormation and other related cloud vendor tooling enable engineers to write declarative code to define a required state of the infrastructure. The tools parse the declarative configuration and take appropriate action to enact the specified state, for example, calling SDKs and APIs, verifying results, iterating etc. ● Pulumi is an open source framework that enables engineers to define IaC using general purpose programming languages, such as Node, Python, .NET Core, and Go. ● Pulumi allows imperative specification of IaC. Engineers can use their favourite language-specific features, idioms, and patterns. The use of language modules, packages, and libraries can also enable code reuse. ● Under the hood, Pulumi transforms code written in the supported languages to a declarative specification model. This model is then used to enact the required infrastructure state. ● Frameworks like Pulumi enable engineers to deploy and configure infrastructure across multiple cloud vendors and services (including Kubernetes clusters). More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dyxVee You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3dyxVee

In this podcast Charles Humble spoke to Dylan Schiemann, co-creator of Dojo and InfoQ’s JavaScript and Web Development lead editor, about the history and current state of Dojo, and key emerging trends in the JavaScript landscape today. Key topics include Dojo’s adoption of Typescript, web components, and client-side libraries such as Svelte and Stencil. Why listen to this podcast: - Modern Dojo (2.0 and upwards) is focussed on being a very small, opinionated reactive framework, but with a lot of the components you need to build a modern JavaScript application built in. - The framework tries to align closely to standards, for example using Web Components extensively for UI components, alongside ES modules and promises. The use of standards, as well as the convergence towards the reactive programming model for web UI, has improved interoperability, though there are some limitations such as the lack of an easy way to share resources across web components. - Dojo was one of the first frameworks to make the decision to switch to Typescript, though it took some time to make that transition. The switch was mainly motivated by TypeScript’s support for interfaces, but it wasn’t until Typescript 2.6 they felt able to ship Dojo 2. - On the client side we’re paying close attention to Svelte and Stencil as two particularly interesting client-side frameworks. - We’ve moved Web Components from early adopter to early majority on the trend report, based on the fact that all browsers accept IE now natively support it, but also large companies such as Apple, Nike and ESPN are deploying web components and their sites. Apple’s iTunes implementation, for example, now uses web components. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Qy75Jr You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Qy75Jr

In this podcast, Daniel Bryant sat down with Gareth Rushgrove, Director of Product Management at Snyk. Topics covered included Kubernetes as a platform, application abstractions, continuous delivery, and implementing good security practices in the cloud native space. Why listen to this podcast: - The value provided by Kubernetes depends on an organisation’s context. Kubernetes acts as both a series of lower-level building blocks for a platform, and also as a very powerful API for deploying and operating container-based applications. - Kubernetes provides several useful abstractions for engineers. For example, Pods, Deployments, and Services. However, Kubernetes doesn’t have an “application”-focused abstraction. Tools such as Helm and specifications like the Cloud Native Application Bundle (CNAB) are driving innovation in this space. - There is a large amount of open source Kubernetes tooling. This has been created by a range of vendors, groups, and individuals. Encouraging this diverse mix of participation is beneficial for the long-term health of the ecosystem. - The Cloud Native Computing Foundation (CNCF) provides a space for people to collaborate regardless of their current organisational affiliations. - Defining appropriate standards within the cloud native space is useful for enabling interoperability and providing common foundations for others to innovate on top of. - Security challenges within IT are socio-technical. Security teams working with cloud native technologies will benefit from continual learning, developing new skills, and researching new tools. For example, the defaults of Kubernetes aren’t necessarily secure, but this can be readily addressed with appropriate configuration. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/38PLPFb You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/38PLPFb

- A Micro frontends is an approach to developing frontends that attempts to take some of the same benefits from Microservices and apply them to frontend development. - Microfront ends can be developed with different technologies and ownership of components on a single view. However, DAZN took a vertical approach to build them. Each Micro frontend is loaded into an app shell that offers an API for crosscutting concerns. Only one Micro frontend is loaded at a time into the app shell. - The ‘Inverse Conway Maneuver’ recommends evolving your team and organizational structure to create the architecture you want. - DAZN derisks deployments by using canaries implemented with Lambda at the Edge on Cloudfront. For code deployments, each of the Micro frontends can be introduced with a limited scope and then expanded once proven stable. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/38BQAC0 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/38BQAC0

In this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows. Why listen to this podcast: - Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage. - There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized. - The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh. Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way. - Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers. The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/39exTWl You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/39exTWl

In this podcast, Daniel Bryant sat down with Brittany Postnikoff, a computer systems analyst specialising on the topics of robotics, embedded systems, and human-robot interaction. Topics discussed included: the rise of robotics and human-robot interaction within modern life, the security and privacy risks of robots used within this context, and the potential for robots to be used to socially engineer people. Why listen to this podcast: - Physical robots are becoming increasingly common in everyday life, for example, offering directions in airports, cleaning the floor in peoples’ homes, and acting as toys for children. - People often imbue these robots with human qualities, and they trust the authority granted to a robot. - Social engineering can involve the psychological manipulation of people into performing actions or divulging confidential information. This can be stereotyped by the traditional “con”. - As people are interacting with robots in a more human-like way, this can mean that robots can be used for social engineering. - A key takeaway for creators of robots and the associated software is the need to develop a deeper awareness of security and privacy issues. - Software included within robots should be patched to the latest version, and any data that is being stored or transmitted should be encrypted. - Creators should also take care when thinking about the human-robot UX, and explore the potential for unintended consequences if the robot is co-opted into doing bad things. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2v5QTav You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2v5QTav

In this podcast, Daniel Bryant sat down with Anurag Goel, Founder and CEO of Render. Topics covered included: the evolution of cloud platforms; simplifying developer experience; running large-scale workloads on Kubernetes; and the future of tooling and platforms within the cloud native computing space. Why listen to this podcast: - Render is aiming to be the next generation of cloud provider. Developers deploy and manage applications via a Platform-as-a-Service (PaaS) -like experience using custom simplified YAML configuration. - Render is built on top of Kubernetes, but the internals and configuration of this orchestration framework is not exposed to end user developers. - Many large scale usages of traditional cloud vendor platforms require the formation of specialised in-house “DevOps” teams. The provision of virtualisation and API-driven operation via the cloud providers was revolutionary, but it didn’t fundamentally change the existing platform paradigm. - Arguably platform usability may have taken a step back with the arrival of public cloud vendor platforms. For example, developers may just want to write code, and not have to write complicated deployment descriptors. Operations team may want to focus on supporting engineers and advising on performance and scale, rather maintaining cloud provisioning scripts. - The Render team are planning to run all future workloads of self-managed Kubernetes, rather than use a hosted offering, due to them experiencing implementation bugs when running their clusters at medium-to-large scale. - The Cloud Native Computing Foundation (CNCF) is encouraging large amounts of innovation within the cloud platform space. However, due to the Cambrian explosion of the Cloud Native Landscape over the past several years, there must surely be consolidation of tools, platforms, and vendors in the near future. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2H2lY1s You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2H2lY1s

In this podcast, Daniel Bryant sat down with Greg Law, CTO at Undo. Topics discussed included: the challenges with debugging modern software systems, the need for “hyper-observability” and the benefit of being able to record and replay exact application execution; and the challenges with implementing the capture of nondeterministic system data in Undo’s LiveRecorder product for JVM-based languages that are Just-In-Time (JIT) compiled. Why listen to this podcast: - Understanding modern software systems can be very challenging, especially when the system is not doing what is expected. When debugging an issue, being able to observe a system and look at logging output is valuable, but it doesn’t always provide all of the information a developer needs. Instead we may need “hyper observability”; the ability to “zoom into” bugs and replay an exact execution. - Being able to record all nondeterministic stimuli to an application -- such as user input, network traffic, interprocess signals, and threading operations -- allows for the replay of an exact execution of an application for debugging purposes. Execution can be paused, rewound, and replayed, and additional logging data can be added ad hoc. - Undo’s LiveRecorder allows for the capture of this nondeterministic data, and this can be exported and shared among development teams. The UndoDB debugger, which is based on the GNU Project Debugger, supports the loading of this data and the execution and debugging in forwards and reverse execution of the application. There is also support for other debuggers, such as that included within IntelliJ IDEA. - Advanced techniques like multi-process correlation reveal the order in which processes and threads alter data structures in shared memory, and thread fuzzing randomizes thread execution to reveal race conditions and other multi-threading defects. - The challenges of using this type of technology when debugging (micro)service-based application lies within the user experience i.e. how should the multiple process debugging experience be presented to a developer? Live Recorder currently supports C/C++, Go, Rust, Ada applications on Linux x86 and x86_64, with Java support available in alpha. Supporting the capture and replay of data associated with JVM language execution, which contain extra abstractions and are often Just-In-Time (JIT) compiled, presented extra challenges. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37XLUa0 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37XLUa0

Today on The InfoQ Podcast, Wes Reisz speaks with CEO and founder of Solo Idit Levine. The two discuss the Three Pillars of Solo around Gloo, their API gateway, interoperability of service meshes (including the work on Service Mesh Interface), and on extending Envoy with Web Assembly (and the recently announced Web Assembly Hub). Why listen to this podcast: - Gloo is a Kubernetes-native ingress controller and API gateway. It’s built on top of Envoy and at its core is open source. - The Service Mesh Interface (SMI) is a specification for service meshes that runs on Kubernetes. It defines a common standard that can be implemented by a variety of providers. The idea of SMI is it’s an abstraction on top of service meshes, so that you can use one language to configure them all. - Autopilot is an open-source Kubernetes operator that allows developers to extend a service mesh control plane. - Lua has been commonly used to extend the service mesh data plane. Led by Google and the Envoy community, web assembly is becoming the preferred way of extending the data plane. Web assembly allows you to write Envoy extensions in any language while still being sandboxed and performant. - WebAssembly Hub is a service for building, deploying, sharing, and discovering Wasm extensions for Envoy. - Wasme is a docker like an open-source commandline tool from Solo to simplify the building, pushing, pulling, and deploying Envoy Web Assembly Filters. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37sYIoE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37sYIoE

Today, on The InfoQ Podcast, Wes Reisz talks with Gunnar Morling. Gunnar is a software engineer at RedHat and leads the Debezium project. Debezium is an open-source distributed platform for change data capture (CDC). On the show, the two discuss the project and many of its use cases. Additionally, topics covered on the podcast include bootstrapping, configuration, challenges, debugging, and operational modes. The show wraps with long term strategic goals for the project. Why listen to this podcast: - CDC is a set of software design patterns used to react to changing data in a data store. Used for things like internal changelogs, integrations, replication, and event streaming, CDC can be implemented leveraging queries or against the DB transaction log. Debezium leverages the transaction log to implement CDC and is extremely performant. - Debezium has mature source and sink connectors for MySQL, SQL Server, and MongoDB. In addition, there are Incumbating connectors for Cassandra, Oracle, and DB2. Community sink connectors have been created for ElasticSearch. - In a standard deployment, Debezium leverages a Kafka cluster by deploying connectors into Kafka Connect. The connectors establish a connection to the source database and then write changes to a Kafka topic. - Debezium can be run in embedded mode. Embedded mode imports Java library into your own project and leverages callbacks for change events. The library approach allows Debezium implementations against other tools like AWS Kinesis or Azure's Event Hub. Going forward, there are plans to make a ready-made Debezium runtime. - Out of the box, Debezium has a one-to-one mapping between tables and Kafka topic queues. The default approach exposes the internal table structure to the outside. One approach to address exposing DB internals is to leverage the Outbox Pattern. The Outbox Pattern uses a separate outbox table as a source. Inserts into your normal business logic tables also make writes to the outbox. Change events are then published to Kafka from the outbox source table. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3737GZB You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3737GZB

In this podcast, Daniel Bryant sat down with Kelsey Hightower, Staff Developer Advocate at Google. Topics covered included: the extensibility of Kubernetes, and why it has become the platform that other platforms are being built on top of; creating event-driven architectures and deploying these onto Function-as-a-Service (FaaS) platforms like the Kubernetes-based Knative and Google Cloud Run; and the benefits of learning, sharing knowledge, and building communities. Why listen to this podcast: - Kubernetes is a platform for building platforms. It may not be as opinionated as traditional Platform-as-a-Service (PaaS) offerings, but it has become popular due to its extensibility. There are PaaS-like solutions built on top of Kubernetes, such as OpenShift, Knative, and Cloud Run. - The creation of common interfaces within Kubernetes -- such as Custom Resource Definitions (CRDs), Container Networking Interface (CNI), and Container Runtime Interface (CRI) -- enabled the adoption of the platform by vendors and the open source community without everyone needing to agree on exactly how to implement extensions. - Although not every workload can be effectively implemented using an event-driven architecture, for those that can the Kubernetes-based Function-as-a-Service (FaaS) platforms like Knative and Cloud Run can handle a lot of the operational management tasks for developers. - Engineers may be able to get ~90% of the “service mesh” traffic management functionality they need from using a simple proxy. However, the separation of the control and data planes within modern service meshes, in combination with the rise in popularity of the sidecar deployment model, has provided many benefits within Kubernetes. - A lot of learning within software development and information technology is transferable. If you spend time going deep in a technology when you begin your career, much of what you learn will be useful when you come to learn the next technology. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/30alHC1 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/30alHC1

In this podcast, Daniel Bryant sat down with Katie Gamanji, Cloud Platform Engineer at Condé Nast International. Topics covered included: exploring the architecture of the Condé Nast Kubernetes-based platform; the importance of enabling self-service deployment for developers; and how the Kubernetes’ Federation API and Cluster API may enable more opportunities for platform automation. - Founded in the early 1900s, Condé Nast is a global media company that has recently migrated their application deployment platforms from individually-curated geographically-based platforms, to a standardised distributed platform based on Kubernetes and AWS. - The Condé Nast engineering team create and manage their own Kubernetes clusters, currently using CoreOS’s/Red Hat’s Tectonic tool. Self-service deployment of applications is managed via Helm Charts. - The platform team works closely with their “customer” developer teams in order to ensure their requirements are being met. - The Kubernetes Federation API makes it easy to orchestrate the deployment of applications to multiple clusters. This works well for cookie-cutter style deployments that only require small configuration differences, such as scaling the number of running applications based on geographic traffic patterns. - The Cluster API is a Kubernetes project to bring declarative APIs to cluster creation, configuration, and management. This enables more effective automation for cluster lifecycle management, and may provide more opportunities for multi-cloud Kubernetes use. - The Condé Nast platform Kubernetes Ingress is handled by Traefik, due to the good Helm support and cloud integration (for example, AWS Route 53 and IAM rule synchronization). The platform team is exploring the use of service mesh for 2020. - Abstractions, interfaces, and security will be interesting focal points for improvement in the Kubernetes ecosystem in 2020. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2FeYPrE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2FeYPrE

In this podcast, Daniel Bryant spoke to Joseph Jacks, Founder of OSS Capital and the Open Core Summit, and discussed topics including the open source and open core models, innovations within open source hardware and the RISC-V instruction set architecture, and current opportunities for disruption using commercial open source software. Why listen to this podcast: - Recently, open source software and the open core business model have driven a lot of innovation and created a lot of value, particularly within the cloud “as-a-service” space. - There has been some disagreement between the open source and commercially-focused communities, for example, in relation to the licencing models and how value is captured. - The Open Core Summit (OCS) is a new conference focusing on the intersection of commercialisation and open source software that aims to facilitate discussion in this space. - Organisations building around open source software can potentially look at large cloud vendors as partners. Public clouds can provide effective distribution, and typically focus on offering breadth of services rather than the depth of expertise that can be provided by a specialist company. - RISC-V is an open-source hardware instruction set architecture (ISA) based on the well-established reduced instruction set computer (RISC) principles. Leveraging RISC-V can reduce the time and cost of customising chip designs. - A lot of recent open source innovation has focused on the infrastructure layer within computing systems. This means that the application layer is now potentially ripe for disruption via commercial open source software. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2rDfYYU You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2rDfYYU

In this special year-end wrap-up podcast Wes Reisz, Shane Hastie, Daniel Bryant, and Charles Humble discuss what we’ve seen in 2019 and speculate a little on what we hope to see in 2020. Topics include business agility and Teal, what it means to be an ethical engineer, bringing your whole self to work, highlights from QCon and InfoQ during 2019, the rise of Python, and progress in quantum computing. Why listen to this podcast: * Business agility is one of the major themes that the InfoQ team has seen emerge this year, with stronger emphasis on outcomes over outputs. We’ve also seen a growing interest in ethics and the ethical implications of the work we all do. * On the programming languages front the rise of Python continues, driven largely by its popularity in data science. * As Kubernetes cements its dominant position we’re hoping to see a simplification of the workflows associated with it, as well as in areas like observability. * There have been several big announcements in quantum computing in the past year, and this is an area we continue to watch with interest. * Another key trend for next year is edge computing. The edge of the cloud infrastructure has an amazing amount of available compute resource, as does the device edge. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Z0Q9OI You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Z0Q9OI

Josh Wills, a software engineer working on data engineering problems at Slack, discusses the Slack data architecture and how they build and observe their pipelines. Josh, along with color commentary such as the move from IC to manager (and back), discusses recommendations, tips, tools, and lessons Slack engineering teams discovered while building products like Slack Search. The podcast covers machine learning, observability, data engineering, and general practices for building highly resilient software. Why listen to this podcast: - Slack has a philosophy of building only what they need. They have a don’t reinvent the wheel mindset. - Slack was originally a PHP monolith. Today, it is largely Hack-lang, HHVM, and several Java and Go binarys. On the data side, application logs are in Thrift (there is a plan to migrate to protobuf). Events are processed through a Kafka cluster that handles 100,000s of events per second. Everything is kept in S3 with a large Hive metastore. EMR is spun up on demand. Presto, Airflow, Slack, Snowflake (business analytics), Quiver (key value store) are all used. - ML worked best for Slack when it was used to help people answer questions. Things like Learn to Rank (LTR) become the most effective use of ML for Slack. - You can get pretty far with rules. Use machine learning when that’s all that’s left. - When you start applying observability to your data pipeline, a key lesson for Slack was to really focus on structured data, tracing, high cardinality events. This let them really use the tools they were already familiar with (ELK, Prometheus, Grafana) and go deep into understanding what’s happening in the systems. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PsVA4q You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2PsVA4q

In this podcast, Daniel Bryant sat down with Bryan Liles, senior staff engineer at VMware. Topics covered included: the challenges with deploying applications into Kubernetes, using the open source Octant tool to increase a user’s understanding of Kubernetes clusters, and how “serverless” technologies may influence the future approaches to building software. Why listen to this podcast: - Octant is a highly extensible platform for developers to better understand the complexity of Kubernetes clusters. Octant runs locally, using the local Kubernetes credentials. It currently displays information about a Kubernetes cluster and related applications as a web page. Soon this tool and resulting display will be provided as a standalone application. - The goal of Octant is to enable users to discover what they need to discover. The tool aims to provide context relevant to where a user is and what they are trying to achieve. The Octant plugin system allows integration with other tooling, such as logging and metrics frameworks. This aims to facilitate quick problem detection and resolution. - Cloud native platforms like Kubernetes are complicated, as there are lots of moving parts. The most important challenge to be tackled to increase the adoption of platforms like Kubernetes is “how do we move code from our IDEs to wherever it needs to run with the least amount of friction?”. Testing needs to be implicit, as does security verification, and the acts of deployment. Kubernetes needs its “Ruby on Rails” moment. - Creating “serverless” systems is an interesting approach, but we may currently be using this technology in a non-optimal way. For example, creating web applications using this technology enables scalability, but can lead to the creation of difficult to understand systems that also require a lot of boilerplate configuration. Arguably, a more interesting use case is implementing large-scale batch processing using simple event-driven models. - The Cloud Native Computing Foundation (CNCF) has created a series of communities of practice called Special Interest Groups (SIGs), such as SIG App Delivery. This allows folks with similar interests to work together as a community, focusing on solving a specific set of well-scoped problems. There are many ways to get involved, from discussions, to coding and creating documentation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37iUwIG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37iUwIG

Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. On today’s podcast, Wes and Victor talk about the realities of building machine learning in the browser. The two discuss the capabilities, limitations, process, and realities around using TensorFlow.js. The two wrap discussing techniques like Model distillation that may enable machine learning models to be deployed in smaller footprints like serverless. - While there are limitations in running machine learning processes in a resource constrained environment like the browser, there are tools like TensorFlow.js that make it worthwhile. One powerful use case is the ability to protect the privacy of a user base while still making recommendations. TensorFlow.js takes advantage of the WebGL library for its more computational intense operations. - TensorFlow.js enables workflows for training and scoring models (doing inference) purely online, by importing a model built offline with more tradition Python tools, and a hybrid approach that builds offline and finetunes online. To build an offline model, you can build a model with TensorFlow Python (perhaps using a GPU cluster). The model can be exported into the TensorFlow SaveModel Format (or the Keras Model Format) and then converted with TensorFlow.js into the TensorFlow Web Model Format. At that point, the can be directly imported into your JavaScript. - TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models and was made available by the Google AI team. It can give developers a quick jumpstart into using trained models. - Model compression promises to make models small enough to run in places we couldn’t run models before. Model distillation is a process where a smaller model is trained to replicate the behavior of a larger one. In one case, BERT (a library almost 500MB in size) was distilled to about 7MB (almost 60x compression). More on this: Quick scan our curated show notes on InfoQ https://bit.ly/32rWnab You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/32rWnab

In this podcast, Daniel Bryant spoke to Michelle Krejci, service engineer lead at Pantheon, about the Drupal and Wordpress webops-based company’s move to a microservices architecture. Michelle is a well-known conference speaker in the space of technical leadership and continuous integration, and she shared her lessons learned over the past four years of the migration. Why listen to this podcast: - The backend for the Pantheon webops platform began as a Python-based monolith with a Cassandra data store. This architecture choice initially enabled rapid feature development as the company searched for product/market fit. However, as the company found success and began scaling their engineering teams, the ability to add new functionality rapidly to the monolith became challenging. - Conceptual debt and technical debt greatly impact the ability to add new features to an application. Moving to microservices does not eliminate either of these forms of debt, but use of this architectural pattern can make it easier to identify and manage the debt, for example by creating well-defined APIs and boundaries between modules. - Technical debt -- and the associated engineering toil -- is real debt, with a dollar value, and should be tracked and made visible to everyone. Establishing “quick wins” during the early stages of the migration towards microservices was essential. Building new business-focused services using asynchronous “fire and forget” event-driven integrations with the monolith helped greatly with this goal. - Using containers and Kubernetes provided the foundations for rapidly deploying, releasing, and rolling back new versions of a service. Running multiple Kubernetes namespaces also allowed engineers to clone the production namespace and environment (without data) and perform development and testing within an individually owned sandboxed namespace. - Using the Apollo GraphQL platform allowed schema-first development. Frontend and backend teams collaborated on creating a GraphQL schema, and then individually built their respective services using this as a contract. Using GraphQL also allowed easy mocking during development. Creating backward compatible schema allowed the deployment and release of functionality to be decoupled.

In today’s podcast we sit down with Ryan Kitchens, a senior site reliability engineer and member of the CORE team at Netflix. This team is responsible for the entire lifecycle of incident management at Netflix, from incident response to memorialising an issue. Why listen to this podcast: - Top level metrics can be used as a proxy for user experience, and can be used to determine that issue should be alerted on an investigated. For example, at Netflix if the customer playback initiation “streams per second” metric declines rapidly, this may be an indication that something has broken. - Focusing on how things go right can provide valuable insight into the resilience within your system e.g. what are people doing everyday that helps us overcome incidents. Finding sources of resilience is somewhat “the story of the incident you didn’t have”. - When conducting an incident postmortem, simply reconstructing an incident is often not sufficient to determine what needs to be fixed; there is no root cause with complex socio-technical systems as found at Netflix and most modern web-based organisations. Instead, teams must dig a little deeper, and look for what went well, what contributed to the problem, and where are the recurring patterns. - Resilience engineering is a multidisciplinary field that was established in the early 2000s, and the associated community that has emerged is both academic and deeply practical. Although much resilience engineering focuses on domains such as aviation, surgery and military agencies, there is much overlap with the domain of software engineering. - Make sure that support staff within an organisation have a feedback loop into the product team, as these people providing support often know where all of the hidden problems are, the nuances of the systems, and the workarounds. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2LLwk8T You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2LLwk8T

In this podcast we sit down with Oliver Gould, co-founder and CTO of Buoyant. Oliver has a strong background in networking, architecture and observability, and worked on solving associated technical challenges at both Yahoo! and Twitter. Oliver is a regular presenter at cloud and infrastructure conferences, and alongside his co-founder William Morgan, you can often find them in the hallway track, waxing lyrical about service mesh -- a term they practically coined -- and trying to bring others along on the journey. Service mesh technology is still young, and the ecosystem is still very much a work in progress, but there have been several recent interesting developments within this space. One of these was the announcement of the service mesh interface (SMI) at the recent KubeCon EU in Barcelona. The SMI spec seeks to unlock service mesh integrators and implementers, as this can provide an abstraction that removes the need to bet on any single service mesh implementation. This can be good for both tool makers and enterprise early adopters. Many organisations like Microsoft and HashiCorp are involved with working alongside the community to help define the SMI, including Buoyant. In this podcast we summarise the evolution of the service mesh concept, with a focus on the three pillars: visibility, security, and reliability. We explore the new traffic “tap” feature within Linkerd that allows near real time in-situ querying of metrics, and discuss how to implement network security by leveraging the primitives like Service Account provided by Kubernetes. We also discuss how reliability features, such as retries, time outs, and circuit-breakers are becoming table stakes for infrastructure platforms. We also cover the evolution of the service mesh interface, explore how service meses may impact development and platforms in the future, and briefly discuss some of the benefits offered by the Rust language in relation to building a data plane for Linkerd. We conclude the podcast with a discussion of the importance of community building. Why listen to this podcast: - A well-implemented service mesh can make a distributed software system more observable. Linkerd 2.0 supports both the emitting of mesh telemetry for offline analysis, and also the ability to “tap” communications and make queries dynamically against the data. The Linkerd UI currently makes use the tap functionality. - Linkerd aims to make the implementation of secure service-to-service communication easy, and it does this by leveraging existing Kubernetes primitives. For example, Service Accounts are used to bootstrap the notion of identity, which in turn is used as a basis for Linkerd’s mTLS implementation. - Offering reliability is “table stakes” for any service mesh. A service mesh should make it easy for platform owners to offer fundamental service-to-service communication reliability to application owners. - The future of software development platforms may move (back) to more PaaS-like offerings. Kubernetes-based function as a service (FaaS) frameworks like OpenFaaS and Knative are providing interesting features in this space. A service mesh may provide some of the glue for this type of platform. - Working on the service mesh interface (SMI) specification allowed the Buoyant team to sit down with other community members like HashiCorp and Microsoft, and share ideas and identify commonality between existing service mesh implementations. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2m5DSJ6 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2m5DSJ6

Today on the podcast, Bernd Rucker of Camunda talks about event sourcing. In particular, Wes and Bernd discuss thoughts around scalability, events, commands, consensus, and the orchestration engines Camunda implemented. This podcast is a primer on considerations between an RDBMS and event-driven systems. Why listen to this podcast: - An event-driven system is a more modern approach to building highly scalable systems. - An RDBMS system can limit throughput in scalability. Camunda was able to achieve higher levels of scale by implementing an event-driven system. - Command and events are often confused. Commands are actions that request something to happen. Events describe something that happened. Confusing the two causes confusion in application development of event-driven systems.

In this podcast we discuss a holistic approach to technical leadership, and Pat provides guidance on everything from defining target operating models, cultivating culture, and supporting people in developing the career they would like. There are a bunch of great stories, several book recommendations, and additional resources to follow up on. * Cultivating organisational culture is much like gardening: you can’t force things, but you can set the right conditions for growth. The most effective strategy is to communicate the vision and goals, lead the people, and manage the systems and organisational structure. * N26, a challenger bank based in Berlin has experienced hypergrowth over the past two years. Both the number of customers and the amount of employees have increased over threefold. This provides lots of opportunities for ownership of product and projects, and it creates unique leadership challenges. * A target operating model (TOM) is a blueprint of a firm's business vision that aligns operating capacities and strategic objectives and provides an overview of the core business capabilities, internal factors, and external drivers, strategic and operational levers. This should be shared widely within an organisation * Pat has curated a “trident operating” model for employee growth. In addition to the class individual contributor (IC) and management tracks, he believes that a third “technical leadership” track provides many benefits. * People can switch between these tracks as their personal goals change. However, this switch can be challenging, and an organisation must support any transition with effective training. * Pat recommends the following books for engineers looking to make the transition to leadership: The Manager’s Path, by Camille Fournier; Resilient Management, by Lara Hogan; Elegant Puzzle, by Will Larson; and Leading Snowflakes by Oren Ellenbogen. Pat has also written his own book, Talking with Tech Leads. * It is valuable to define organisation values upfront. However, these can differ from actual culture, which more about what behaviours you allow, encourage, and stop. * Much like the values provided by Netflix’s Freedom and Responsibility model, Pat argues that balancing autonomy and alignment within an organisation is vital for success. Managers can help their team by clearly defining boundaries for autonomy and responsibility. * Developing the skills to influence people is very valuable for leaders. Influence is based on trust, and this must be constantly cultivated. Trust is much like a bank account, if you don’t regular deposit actions to build trust, you may find yourself going overdrawn when making a deposit. This can lead to bad will and defensive strategies being employed.

Cilium is open source software for transparently securing the network connectivity between application services deployed using Linux container management platforms like Docker and Kubernetes. It is a CNI plugin that offers layer 7 features typically seen with a service mesh. On this week’s podcast, Thomas Graf (one of the maintainers of Cilium and co-founder of Isovalent) discusses the recent 1.6 release, some of the security questions/concerns around eBPF, and the future roadmap for the project. Why listen to this podcast: * Cilium brings eBPF to the Cloud Native World. It works across both layer 4 and a layer 7. While it started as a pure eBPF plugin, they discovered that just caring about ports was not enough from a security perspective. * Cilium went 1.0 about a year and a half ago. 1.6 is the most featured-packed release of Cilium yet. Today, it has around 100 contributors. * While Cilium can make it much easier to manage IPTables, Cilium overlaps with a service mesh in that it can do things like understand application protocols, HTTP routes, or even restrict access to specific tables in data stores. * Cilium provides both in kernel and sidecar deployments. For sidecar deployments, it can work with Envoy to switch between kernel space and userspace code. The focus is on flexibility, performance, and low overhead. * BPF (Berkeley Packet Filter) was initial designed to do filtering on data links. eBPF has the same roots but it’s now used for system call filtering, tracing, sandbox, etc. It’s grown to be a general-purpose programming language to extend the Linux kernel. * Cilium has a multi-cluster feature built-in. The 1.6 release can run in a kube-proxy free configuration. It allows fine-grain network policies to run across multiple clusters without the use of IPTables. * Cilium offers on-the-wire encryption using in-kernel encryption technology that enables mTLS across all traffic in your service fleet. The encryption is completely transparent to the application. * eBPF has been used in all production environments at Facebook since May 2017. It’s been used at places like Netflix, Google, and Reddit. There are a lot of companies who have an interest in eBPF being secure and production-ready, so there’s a lot of attention focused on fixing and resolving and security issues that arise. * 1.6 also released KVstore-free operation, socket-based load balancing, CNI chaining, Native AWS ENI mode, enhancements to transparent encryption, and more. * The plans for 1.17 is to keep raising up the stack into the socket level (to offer things like load balancing and transparent encryption at scale) and likely offering deeper security features such as process-aware security policies for internal pod traffic. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2HCGnLa You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2HCGnLa

The three pillars of observability are logs, metrics, and tracing. Most teams are able to handle logs and metrics, while proper tracing can still be a challenge. On this podcast, we talk with Yuri Shkuro, the creator of Jaeger, author of the book Mastering Distributed Tracing, and a software engineer at Uber, about how the Jaeger tracing backend implements the OpenTracing API to handle distributed tracing. Why listen to the podcast: - Jaeger is an open-source tracing backend, developed at Uber. It also has a collection of libraries that implement the OpenTracing API. - At a high level, Jaeger is very similar to Zipkin, but Jaeger has features not available in Zipkin, including adaptive sampling and advanced visualization tools in the UI. - Tracing is less expensive than logging because data is sampled. It also gives you a complete view of the system. You can see a macro view of the transaction, and how it interacted with dozens of microservices, while still being able to drill down into the details of one service. - If you have only a handful of services, you can probably get away with logging and metrics, but once the complexity increases to dozens, hundreds, or thousands of microservices, you must have tracing. - Tracing does not work with a black box approach to the application. You can't simply use a service mesh then add a tracing framework. You need correlation between a single request and all the subsequent requests that it generates. A service mesh still relies on the underlying components handling that correlation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZlvMyR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZlvMyR

ROS is the Robotic Operating System. It’s been used by thousands of developers to prototype and create a robotic application. ROS can be found on robotics in warehouses, self-driving car companies, and on the International Space Station. Louise Poubel is an engineer working with Open Robotics. Today on the podcast, she talks about what it takes to develop software that moves in physical space, including the Sense, Think, Act Cycle, the developer experience, and architecture of ROS. Why listen to this podcast: - Writing code for robot development, you use the Sense, Think, Act Cycle. - ROS is an SDK for robotics. It provides a communication layer that enables data to flow between nodes that handle sensors, logic, and actuation. - ROS has two versions and has been around for twelve years. ROS 1 was entirely implemented in C. ROS 2 offers is a common C layer with implementations in many different languages, including Java, JavaScript, and Rust. - Released on a six-month cadence, Dashing was the latest release (May 2019). Previous releases were supported for one year, Dashing is the first LTS and will be supported for two years. - ROS 2 builds on top of the standard Data Distribution Service (DDS) that you find in mission-critical systems like nuclear power and airplanes. - Simulation is an important step in robotics. It allows you to prototype a system before deploying to a physical system. - Rviz is a three-dimensional visualizer used to visualize robots, the environments they work in, and sensor data. It is a highly configurable tool, with many different types of visualizations and plugins. It allows you to put together all your data in one place and see it. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZbOvrO You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZbOvrO

In this podcast we sit down with Matt Klein, software plumber at Lyft and creator of Envoy, and discuss topics including the continued evolution of the popular proxy, the strength of the open source Envoy community, and the value of creating and implementing standards throughout the technology stack. We also explore the larger topic of cloud natives platforms, and discuss the tradeoffs between using a simple and opinionated platform against something that is bespoke and more configurable, but also more complex. Related to this, Matt shares his thoughts on when and how to make the decision within an organisation to embrace technology like container orchestration and service meshes. Finally, we explore the creation of the new Envoy Mobile project. The goal of this project is to expand the capabilities provided by Envoy all the way out to mobile devices powered by Android and iOS. For example, most current user-focused traffic shifting that is conducted at the edge is implemented with coarse-grained approaches via by BGP and DNS, and using something like Envoy within mobile app networking stacks should allow finer-grained control. Why listen to this podcast: - The Envoy Proxy community has grown from strength-to-strength over the last year, from the inaugural EnvoyCon that ran alongside KubeCon NA 2018, to the increasing number of code contributions from engineers working across the industry - Attempting to create a community-driven “universal proxy data plane” with clearly defined APIs, like Envoy’s XDS API, has allowed vendors to collaborate on a shared abstraction while still allowing room for “differentiated success” to be built on top of this standard Google’s gRPC framework is adopting the Envoy XDS APIs, as this will allow both Envoy and gRPC instances to be operated via a single control plane, for example, Google Cloud Platform’s Traffic Director service. - There is a tendency within the software development industry to fetishise architectures that are designed and implemented by the unicorn tech companies, but not every organisation operates at this scale. - However, there has also been industry pushback against the complexity that modern platform components like container orchestration and service meshes can introduce to a technology stack. - Using a platform within these components provides the best return on investment when an organisation’s software architecture and development teams have reached a certain size. - Function-as-a-Service (Faas)-type platforms will most likely be how engineers will interact with software in the future. Business-focused developers often do not want to interact with the platform plumbing Envoy Mobile is building on prior art, and aims to expand the capabilities provided by Envoy all the way out to mobile devices using Android and iOS. Most current end user traffic shifting is implemented with coarse-grained approaches via BGP and DNS, and using something like Envoy instead will allow finer-grained control. - Using Envoy Mobile in combination with Protocol Buffers 3, which supports annotations on APIs, can facilitate working with APIs offline, configuring caching, and handling poor networking conditions. One of the motivations for this work is that small increases in application response times can lead to better business outcomes. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/33nlGMu You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/33nlGMu

On this podcast, we’re talking to Armon Dadgar, co-founder and CTO of HashiCorp. Alongside Mitchell Hashimoto, Armon founded HashiCorp over six years ago, and the company has gone from strength to strength, with their open source infrastructure product suite now consisting of Consul, Nomad, Vault and Terraform. We discuss the formation of the HashiCorp research division, and explore some of the computer science research underpinning Consul and Nomad. We also cover the challenges of supporting teams when they are looking to embrace new modes of working with dynamic infrastructure, and Armon introduces the new learn.hashicorp.com educational website and accompanying community and support forums. Why listen to this podcast: - There is a lot of fundamental computer science research that underpins the HashiCorp infrastructure workflow and configuration tooling. This helps to ensure that these mission-critical tools perform as expected, and enables sound reasoning about scaling these technologies. - The HashiCorp founders recognised the value of creating an industrial research-focused department within the company even when there were only 30 staff. - The Consul service mesh and distributed key value store leverages consensus and gossip algorithms from computer science research, Raft and SWIM, respectively. The HashiCorp team contributed a novel research-based improvement to SWIM -- Lifeguard: SWIM-ing with Situational Awareness -- that was presented at the DSN academic conference - Initially HashiCorp produced a new tool every 6-12 months, focusing on filling gaps within the infrastructure workflow tooling market. Now the focus is on refining the operator/user experience of the existing tools, creating more integrations with other platforms and tooling, and facilitating engineering teams adopting these tools, via the creation of educational resources and community forums. - Standardisation within computing technology can offer many benefits, especially where interoperability is required or technology switching costs are high. Care must be taken to ensure the correct interfaces are created, and that the time is right to create appropriate abstractions. - The HashiCorp team are focusing on "marching up the stack", with the goal that a lot of the underlying "plumbing" should be hidden from, or easily configurable by, application developers. This will allow developers to focus on adding value related to their business or organisation, rather than getting stuck with managing infrastructure. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2KptB3d You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2KptB3d

In this episode recorded at QCon London 2019 Shane Hastie, Lead Editor for Culture & Methods, first spoke to Kingsley Davies about ethics and then with Cat Swetel about requisite variety and being mindful of the impact our decisions have for the future. Why listen to this podcast: • The need to explore the application of technology for good • The need for ethical standards in the technology industry • Data is the new oil and it is frequently used in ways that are not in the best interest of society • Other engineering professions have codes of conduct and ethical frameworks that are mandated as part of the education process, software engineering currently has very little • Ashby’s law of requisite variety – the more options that are available to a system, the more resilient the system is applies to all aspects of our socio-technical systems • We exist in the realm of ethics – we can’t just go to work and do what you’re told. Everything we do is a choice and our choices have a huge impact on the future

The promise of Java has always been, “write once, run anywhere.” This was enabled through just-in-time compilation, which allowed developers to target a platform at compilation. But, this flexibility has given rise to comments like, “Java is slow.” What if you could compile Java to Native Code? On this podcast, we’re talking to Thomas Wuerthinger, a senior research director at Oracle Labs. Leading programming language implementation teams for Java, JavaScript, Ruby, and R. He is the architect of the Graal compiler and the Truffle self-optimizing runtime. Why listen to the podcast: - The GraalVM project was initially just a replacement for the JVM C2 just-in-time compiler, but has evolved to include support for multiple languages, as well as an ahead-of-time compiling mode. - Support for multiple languages can provide better performance for some languages, as well as making direct calls without inter-process communication. - With GraalVM’s AOT compilation, you can statically link system libraries, which allows you to run a static binary on a bare-metal Docker image, without even a Linux distribution. - The major benefits of AOT are minimized startup time, memory footprint, and packaging size. This can come with a trade-off in reduced maximum throughput and higher latency. - The GraalVM roadmap includes supporting additional platforms, such as Windows and mobile, as well as performance improvements for both the JIT and AOT compilers. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Y2hPk2 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Y2hPk2

On this podcast, Wes talks to John Xmas. Johnny works for Kasada, a company that offers a security platform to help ensure only your users are logging into your web applications. Johnny is a well-known figure in the security space. The two discuss common attack vectors, the OWASP Top 10, and then walk through what hackers commonly do attempting to compromise a system. The show is full of advice on protecting your systems including topics around Defense in Depth, Time-Based Security, two-factor authentication, logging/alerting, security layers, and much more. Why listen to this podcast: - While there are sophisticated web attacks out there that use things like PhantomJS or Headless Chome, the vast majority of the web application attacks are the same unsophisticated scripted attacks that you always hear about. These are simple scripts using tools like curl and BurpSuite with Python or JavaScript. These simple scripts are still incredibly effective. - OWASP Top 10 really hasn’t changed all that much in the last ten years. For example, despite being the number one approach used to educate defensive engineers on how to protect their apps, SQLI (SQL Injection) is still the most common attack. We continue to repeat the same mistakes that have exposed systems for a decade now. - Phishing is by and far the quickest way to compromise a system. Defensive in Depth, security boundaries, limiting local admin rights are all things that corporations can implement to minimize the blast radius. - Attackers have hundreds of gigs of actual username/password combinations that have been exposed from all the breaches over the past few years. These are often a first step when attempting to compromise a system. It’s more often likely that they will figure out a valid email pattern for a company and then feed actual names into that pattern to go after the username. From there, brute force attacks with those usernames against libraries of passwords is a common approach. - A common approach is to go after an email login. While the email can be a treasure trove of information, it’s more about using those credentials in other places. It’s pretty common, for example, to use those credentials to get into a network with a VPN. - Captcha/reCaptcha is not very effective and preventing these brute force attacks. There are a large number of bypasses and even Mechanical Turk companies that are available to bypass these tools. What can be effective is Time Based Security because it slows the attackers down. If you can slow them down, you can make the attack say long to succeed that they’ll go somewhere else. - Once inside the network, most companies often have little security on internal systems. Multi-factor authentication, not just on the front door, but on internal systems is a huge step in the right direction. Monitoring not only for failed login attempts but, in some situations, valid login attempts (such as when a domain admin logs into a domain controller) should absolutely be used. - When it comes to application security between services within a network, the best advice is to make sure developers really understand what is trying to be accomplished by something like JWT (JSON Web Tokens). Often its the lack of understanding of what they’re actually doing that leads to system vulnerabilities. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2MSIAXG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2MSIAXG

Today on the podcast, Wes talks with Mike Milinkovich, Executive Director of the Eclipse Foundation. The Eclipse Foundation was chosen to govern the evolution of Oracle’s Java EE to Jakarta EE. The two discuss the project, the recent news about issues with the javax namespace, the challenges around bundling a Java Runtime with Eclipse, and the path forward for Jakarta EE 9 and beyond. Why listen to this podcast: - Java EE, unlikely Java SE, has always been a multi-vendor ecosystem. It made sense for everyone for Oracle to invite their partners to be involved in the governance of the specification for Java EE for it to continue moving forward. This is the reason for moving Java EE into the Eclipse Foundation as Jakarta EE. - The current plan is for the Eclipse Foundation to get a copyright license to evolve the text of the specification and not a license to the trademarks of Java EE itself. - The javax namespace must remain as is. For it to be evolved, a different namespace must be used. The javax namespace is a trademark of Oracle. Because of this, there are quality controls that Oracle required for its evolution. Ultimately because of those controls, the Eclipse Foundation felt it was better to branch javax into a different namespace and evolve it separately solely under Jakarta EE governance. - Jakarta EE 8 is targeted to be released around Oracle Code ONE. Jakarta EE 8 will be exactly the same as Java EE 8. * The only difference is it will be licensed from Jakarta, not Oracle and only requires membership in the Working Group. - Beyond EE 8, the release cycle, the plan for moving the javax namespace (and keeping compatibility with both the old javax namespace and the new namespace), and new specifications for inclusion into Jakarta EE are still active areas of discussion. - Unrelated to the discussion of Jakarta EE (but discussed in the same board meeting), an attempt to bundle OpenJ9 with the Eclipse IDE failed because of licensing restrictions around a certified Java Runtime. OpenJ9 is certified when acquired through an IBM channel, but not when downloaded directly for us. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2HSfcfM You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2HSfcfM

Ludwig is a code-free deep learning toolbox originally created and open sourced by UberAI. Today, on the podcast the creator of Ludwig Piero Molino and Wes Reisz discuss the project. The two talk about how the project works, its strengths, it’s roadmap, and how it’s being used by companies inside (and outside) of Uber. They wrap by discussing path ahead for Ludwig and how you can get involved with the project. Why listen to this podcast: • Uber AI is the research and platform team for everything AI at the company with the exception of self-driving cars. Self-driving cars are left to Uber ATG. • Ludwig allows you to specify a Tensorflow model in a declarative format that focuses on your inputs and outputs. Ludwig then builds a model that can deal with those types of inputs and outputs without a developer explicitly specifying how that is done. • Because of Ludwig’s datatype abstraction for inputs and outputs, there is a huge range of applications that can be created. For example, an input could be text and output could be a category. In this case, Ludwig will create a text classifier. An image and text input (such as a question: “Is there a dog in this image”) would output a question answering system. There are many combinations that are possible with Ludwig. • Uber is using Ludwig for text classification for customer support. • Datatypes can be extended easily with Ludwig for custom use cases. • Ludwig would love to have people contribute to the project. There are simple feature requests that are just not prioritized with the current contributor workload. It’s a great place to get involved with machine learning and gain experience with the project. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2JGA5wC You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2JGA5wC From time to time InfoQ publishes trend reports on the key topics we’re following, including a recent one on DevOps and Cloud. So if you are curious about how we see that state of adoption for topics like Kubernetes, Chaos Engineering, or AIOps point a browser to http://infoq.link/devops-trends-2019.

Ben Sigelman is the CEO of Lightstep and the author of the Dapper paper that spawned distributed tracing discussions in the software industry. On the podcast today, Ben discusses with Wes observability, and his thoughts on logging, metrics, and tracing. The two discuss detection and refinement as the real problem when it comes to diagnosing and troubleshooting incidents with data. The podcast is full of useful tips on building and implementing an effective observability strategy. Why listen to this podcast: - If you’re getting woke up for an alert, it should actually be an emergency. When that happens, things to think about include: when did this happen, how quickly is it changing, how did it change, and what things in my entire system are correlated with that change. - A reality that seems to be happening in our industry is that we’re coupling the move to microservices with a move to allowing teams to fully self-determine technology stacks. This is dangerous because we’re not at the stage where all languages/tools/frameworks are equivalent. - While a service mesh offers a great potential for integrations at layer 7 many people have unrealistic expectations on how much observability will be enabled by a service mesh. The service mesh does a great job of showing you the communication between the services, but often the details get lost in the work that’s being done inside the service. Service owners need to still do much more work to instrument applications. - Too many people focus on the 3 Pillars of Observability. While logs, metrics, and tracing are important, observability strategy ought to be more focused on the core workflows and needs around detection and refinement. - Logging about individual transactions is better done with tracing. It’s unaffordable at scale to do otherwise. - Just like logging, metrics about individual transactions are less valuable. Application level metrics such as how long a queue is are metrics that are truly useful. - The problem with metrics are the only tools you have in a metrics system to explain the variations that you’re seeing is grouping by tags. The tags you want to group by have high cardinality, so you can’t group them. You end up in a catch 22. - Tracing is about taking traces and doing something useful with them. If you look at hundreds or thousands of tracing, you can answer really important questions about what’s changing in terms of workloads and dependencies about a system with evidence. - When it comes to serverless, tracing is more important than ever because everything is so ephemeral. Node is one of the most popular serverless languages/frameworks and, unfortunately, also one of the hardest of all to trace. - The most important thing is to make sure that you choose something portable for the actual instrumentation piece of a distributed tracing system. You don’t want to go back and rip out the instrumentation because you want to switch vendors. This is becoming conventional wisdom. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PPIdeE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2PPIdeE

- Web Assembly (wasm) is a set of instructions or a low-level byte code that is a target for higher level languages. It was added to the browser because it was a portion of the web platform that many felt was just missing. - Wasm is still a young technology. It performs really well for computationally intensive applications and also offers performance consistency (because it lacks a garbage collector). - Bootstrapping an application using the Rust toolchain looks like: pull down a template, export a function using an attribute (defines that you want to access this function from JavaScript), and run a tool called wasm-pack (compiles it into Web Assembly and then runs a tool called wasm-bindgen that generated Rust types for Wasm). Then you can talk to that binary as if it was written in JavaScript in your code. - Cloudflare workers allow JavaScript that you might have written for a server to be written and distributed at the application edge (or close to the end user). It uses a similar model as serverless architecture platforms. - Interesting use cases such as A/B testing, DDoS prevention, server-side rendering, or traffic shaping can be done at the edge. - Wasm is an approach to bringing full application experiences to the edge. - Wasi (Web Assembly System Interface) is a standardized interface for running Web Assembly for places that are outside of the web. Fastly recently released a pure Web Assembly runtime for their edge that is built on top of Wasi called Lucet (allows access to lower level things at the edge like sockets and UDP). - Zoom has a web client written in Web Assembly. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Dw3jcH You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Dw3jcH

Bryan Cantrill is the CTO of Joyent and well known for the development of DTrace at Sun Microsystems. Today on the podcast, Bryan discusses with Wes Reisz a bit about the origins of DTrace and then spends the rest of the time discussing why he feels Rust is the “biggest development in systems development in his career.” The podcast wraps with a bit about why Bryan feels we should be rewriting parts of the operating system in Rust. Why listen to the podcast: • DTrace came down to a desire to use Dynamic Program Text Modification to instrument running systems (much like debuggers do) and has its origins to when Bryan was an undergraduate. • When a programming language delivers something to you, it takes it from you in the runtime. The classic example of this is garbage collection. The programming language gives you the ability to use memory dynamically without thinking of how the memory is stored in the system, but then it’s going to exact a runtime cost. • One of the issues with C is that it just doesn’t compose well. You can’t just necessarily pull a library off the Internet and use it well. Everyone’s C is laden with some many idiosyncrasies on how it’s used and the contract on how memory is used. • Ownership is statically tracking who owns the structure. It’s ownership and the absence of GC that allows you to address the composability issues found in C. • It’s really easy in C to have integer overflow which leads to memory safety issues that can be exploited by an attacker. Rust makes this pretty much impossible because it’s very good at how it determines how you use signed vs unsigned types. • You don’t want people solving the same problems over and over again. You want composability. You want abstractions. What you don’t want is where you’ve removed so much developer friction that you develop code that is riddled with problems. For example, it slows a developer down to force them to run a linter, but it results in better artifacts. Rust effective builds a lot of that linter checking into the memory management/type checking system. • While there’s some learning curve to Rust. It’s not that bad if you realize there are several core concepts you need to understand to understand Rust. Rust is one of those languages that you really need to learn in a structured way. Sit down with a book and learn it. • Rust struggles when you have objects that are multiply owned (such as a Doubly Linked List). It’s because it doesn’t know who owns what. While Rust supports unsafe operations, you should resist the temptation to develop with a lot of unsafe operations if you want the benefits of what Rust offers developers. • Firmware is a great spot for growing Rust development in a process of replacing bits of what we think of as the operating system. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2uZ5QHZ You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2uZ5QHZ

Duncan Macgregor speaks with Wes Reisz about the work being done on the experimental Graal Compiler. He talks about the use cases and where the new JIT compiler excels really well (compared to C2). In addition, Duncan talks about the relationship of Graal to Truffle. The two then discuss a language Duncan works on at OracleLabs (TruffleRuby) that is being implemented on the stack. Finally, the podcast wraps with a discussion of Project Loom and its relationship to TruffleRuby and Graal. Why listen to this podcast: - Graal is a replacement for the JVM’s C2 JIT compiler. It was tracked with JEP 295 (Ahead-of-Time Compilation) and included in Java 9. As of Java 10, Graal is experimental for the Linux x64 platform. - Graal is written in Java and excels at implementing code that takes a functional approach to solving problems (such as Scala). It can also offer improvements / optimizations for other languages (including other non-traditional JVM languages such as C and Ruby). - Truffle is a language implementation framework used my Graal. The idea is rather than having to write a compiler for your language, you can write an interpreter. This gives you the ability to write specializations at a higher level of abstraction that yields performance and better understanding. - Truffle’s architecture and design allows things like allowing unrelated languages to do interop, garbage collection, and types. - TruffleRuby and JRuby started off with a lot of shared code. They’ve branched and JRuby today focuses on integration with other Java classes. It compiles to bytecode and then relies on the C2 JIT to run on the JVM. TruffleRuby doesn’t try to compile to Java classes and only uses the Truffle framework to compile the things it needs. TruffleRuby is able to use most of native Ruby. - Project Loom is a project that aims to add one shot delimited continuations to the JVM. It leverages fibers (a much lighter concurrency primitive than threads) and can literally run millions of them.

Today on The InfoQ Podcast, Wes talks with Rod Johnson. Rod is famously responsible for the creation of the Spring Framework. The two talk about the early years of the framework and provides some of the history of its creation. After discussing Spring, Wes and Rod discuss languages he’s been involved with since Java (these include Scala and TypeScript). He talks a bit about what he liked (and didn’t like) about each. Finally, the two wrap by discussing Atomist and how they’re trying to change the idea of software delivery from a statically defined pipeline (located in individual repositories) to an event hub that drives a series of actions for software delivery. He describes this as creating an API for your software. Why listen to this podcast: - The initial origins of the Spring Framework really came about through a process of trying to write a really great book about J2EE in 2002. It was through that process that Rod Johnson found he felt there was a better way and ultimately lead to the creation of the Spring Framework. - What started as examples and references, became the Spring Framework. By 2005 there were about 2 million downloads of the Spring Framework. After leaving VMWare in 2013, Rod spent several years working with Scala. One of the elegant features that really attracted Rod to Scala was how everything is an expression. One of the things he didn’t like was an affinity to overly complex approaches to problem solving. - Today at Atomist, Rod does a lot of work in Node. He really enjoys the robust extra layer of typing over a dynamic language and the ability to escape to JavaScript if needed (similar to escaping types with reflection in Java found in the internals of the Spring Framework). - Atomist, the company he founded after leaving VMWare, is rethinking CI/CD from a static pipeline defined in every repository to an event-driven system that defines how to respond to specific events (such as a push from Git). For example, all pushes with Spring Boot can be configured to be scanned with SonarQube or because a push has kubespec it might get deployed to a K8 cluster. He describes this as creating an API for your software. - One of the reasons Atomist integrates so tightly with Slack (and other similar messaging platforms) is because it allows developers to shape their own relevant messages. By joining (or leaving channels), people are able to subscribe to only the information they actually want. Meeting developers inside Slack is an important interface for Atomist. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2FxK3xf You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2FxK3xf

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Katharine discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Katharine is the Co-Founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynotes at QCon.ai. Why listen to this podcast: - Ethical machine learning is about practices and strategies for creating more ethical machine learning models. There are many highly publicized/documented examples of machine learning gone awry that show the importance of the need to address ethical machine learning. - Some of the first steps to prevent bias in machine learning is awareness. You should take time to identify your team goals and establish fairness criteria that should be revisited over time. This fairness criteria then can be used to establish the minimum fairness criteria allowed in production. - Laws like GDPR in the EU and HIPAA in the US provide privacy and security to users and have legal implications if not followed. - Adversarial examples (like the DolphinAttack that used subsonic sounds to activate voice assistants) can be used to fool a machine learning model into hearing or seeing something that’s not there. More and more machine learning models are becoming an attack vector for bad actors. - Machine learning is always an iterative process. - Zero-Knowledge Computing (or Federated Learning) is an example of machine learning at the edge and is designed to respect the privacy of an individual’s information. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2TD3nSd You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2TD3nSd

Today on The InfoQ Podcast, Wes Reisz speaks with Grady Booch. Grady is well known as the co-creator of UML, an original member of the design patterns movement, and now work he’s doing around Artificial Intelligence. On the podcast today, the two discuss what today’s reality is for AI. Grady answers questions like what does an AI mean to the practice of writing software and around how he seems it impact delivering software. In addition, Grady talks about AI surges (and winters) of over the years, the importance of ethics in software, and host of other related questions. Why listen to this podcast: - There have been prior ages of AI that has lead to immediate winters of where reality set in. It stands to reason, there will be a version of an AI winter that follows today’s excitement around deep learning. - AIs are beginning to look at the code for testing edge cases in software and do things such as looking over your shoulder and identifying patterns in the code that you write. - AIs will remove tedium for software developers; however, software developer is (and will remain) a labor-intensive activity for decades to come.nAI is another bag of tools in a larger systems activity. - Much of the AI developers are young white men from the United States. That has a number of inherent biases in this fact. There are several organizations that are focused on combating some of these biases and bringing ethical learning into the field. This is important for us to be aware of and encourage. - The traditional techniques of systems engineering we know for building non-AI systems will still apply. AI’s are pieces of larger systems. That might be really interesting parts, but it’s just a part of a larger system that requires a lot of non-AI engineering use cases. - Early machine learning systems were mostly learn and forget systems. You teach them, you deploy them, and you walk away. Today, we do continuous learning and we need to integrate these new models into the delivery pipeline. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2SjJOsq You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2SjJOsq

Today on The InfoQ Podcast, Wes talks with Joe Beda. Joe is one of the co-creators of Kubernetes. What started in the fall of 2013 with Craig McLuckie, Joe Beda, and Brendan Burns working on cloud infrastructure has become the default orchestrator for cloud native architectures. Today on the show, the two discuss the recent purchase of Heptio by VMWare, the Kubernetes Privilege Escalation Flaw (and the response to it), Kubernetes Enhancement Proposals, the CNCF/organization of Kubernetes, and some of the future hopes for the platform. Why listen to this podcast: - Heptio, the company Joe and Craig McLuckie co-founded, viewed themselves as not a Kubernetes company, but more of a cloud native company. Joining VMWare allowed the company to continue a mission of helping people decouple “moving to cloud/taking advantage of cloud” patterns (regardless of where you’re running). - Re:Invent 2017 when EKS was announced was a watershed moment for Kubernetes. It marked a time where enough customers were asking for Kubernetes that the major cloud providers started to offer first-class support. - Kubernetes 1.13 included a patch for the Kubernetes Privilege Escalation Flaw Patch. While the flaw was a bad thing, it demonstrated product maturity in the way the community-based security response. - Kubernetes has an idea of committees, sigs, and working groups. Security is one of the committees. There were a small group of people who coordinated the security response. From there, trusted sets of vendors validated and test patches. Most of the response is based on how many other open source projects handle security response. - Over the last couple of releases, Kubernetes has introduced a Sig Architecture special interest group. It’s an overarching review for changes that sweep across Kubernetes. As part of Sig Architecture, the Kubernetes community has introduced Kubernetes Enhancement Proposal process (or KEPs). It’s a way for people to propose architectural changes to Kubernetes. - The goal of the CNCF is to curate and provide support to a set of projects (of which Kubernetes is one). The TOC (Technical Oversight Committee) decides which projects are going to be part of the CNCF and how those projects are supported. - Kubernetes was always viewed by the creators as something to be build on. It was never really viewed as the end goal. You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq

Today on the InfoQ Podcast, Wes speaks with ThirdLove’s Megan Cartwright. Megan is the Director of Data Science for the personalized bra company. In the podcast, Megan first discusses why their customers need a more personal experience and how their using technology to help. She focuses quite a bit of time in the podcast discussing how the team got to an early MVP and then how they did the same for getting to an early machine learning MVP for product recommendations. In this later part, she discusses decisions they made on what data to use, how to get the solution into production quickly, how to update/train new models, and where they needed help. It’s a real early stage startup story of a lean team leveraging machine learning to get to a practical recommendations solution in a very short timeframe. Why listen to this podcast: - The experience for women selecting bras is poor experience characterized by awkward fitting experiences and an often uncomfortable product that may not even fit correctly. ThirdLove is a company built to serve this market. - ThirdLove took a lean approach to develop their architecture. It’s built with the Parse backend. The leveraged Shopify to build the site. The company’s first recommender system used a rules engine embedded into the front end. After that, they moved to a machine learning MVP with a Python recommender service that used a Random Forest algorithm in SciKit-Learn. - Despite having the data for 10 million surveys, the first algorithms only need about 100K records to be trained. The takeaway is you don’t have to have huge amounts of data to get started with machine learning. - To initially deploy their ML solution, ThirdLove first shadowed all traffic through the algorithm and then compared it to what was being output by the rules engine. Using this along with information on the full customer order lifecycle, they validated the ML solution worked correctly and outperformed the rules engine. - ThirdLove’s machine learning story shows that you move towards a machine learning solution quickly by leveraging your own network and using tools that may already familiar to your team. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2G9RnQn You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2G9RnQn

Lynn Langit is a consulting cloud architect who holds recognitions from all three major cloud vendors on her contributions to their respective communities. On today’s podcast, Wes talks with Lynn about a concept she calls 25% time and a project it led her to become involved within genomic research. 25% time is her own method of learning while collaborating with someone else for a greater good. A recent project leads her to become involved with the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia. Through cloud adoption and some lean startup practices, they were able to drop the run time for a machine learning algorithm against a genomic dataset from 500 hours to 10 minutes. Why listen to this podcast: - 25% time is a way to learn, study, or collaborate with someone else for a greater good. It’s unbilled time in the service of offers. Using the idea of 25% time along with some personal events that occurred in her life, Lynn became involved with genomic researchers in Australia. - Price of genomic sequencing has dropped. The price drop has enabled researchers to create huge repositories of genomic data; however, it was mostly on-prem. The idea of building data pipelines was pretty new in the genome community. Additionally, the genome itself is 3 billion data points. A variant of as little at 10-15 variants can be statistically significant. - The challenge was to leverage cloud resources. To gain a quick win and buy-in for Commonwealth Scientific and Industrial Research Organisation (or CSIRO an independent Australian federal government agency) for cloud adoption, a first step was to capture interest in the idea. So the team stored their reference data in the cloud and enabled access via a Jupyter Notebook. - They demonstrated a use case against the genomic data set leveraging a synthetic phenotype (or a fake disease) called hipsterdom. The solution became a basis for global discussion that got more people involved in the community. - By leveraging cloud resources, the CSIRO was able to get a run their dataset that took 500 hours against an on-prem Spark cluster to 10 minutes. - Learning new programming language has unseen benefits. For example, Ballerina (a language written as an integration language between APIs) interested Lynn because of its live visual diagrams; however, benefited her with some of the cloud pipelines because of its ability to produce YAML files. You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2T2LZBQ

In this podcast Charles Humble and Wes Reisz talk about autonomous vehicles, GDPR, quantum computing, microservices, AR/VR and more. * Waymo vehicles are now allowed to be on the road in California running fully autonomous; they seem to be a long way ahead in terms of the number of autonomous miles they’ve driven, but there are something like 60 other companies in California approved to test autonomous vehicles. * It seems reasonable to assume that considerably more regulation around privacy will appear over the next few years, as governments and regulators grapple with not only social media but also who owns the data from technology like AR glasses or self-driving cars. * We’ve seen a huge amount of interest in the ethical implications of technology this year, with Uber getting into some regulatory trouble, and Facebook being co-opted by foreign governments for nefarious purposes. As software becomes more and more pervasive in people's lives the ethical impact of what we all do becomes more and more profound. * Researchers from IBM, the University of Waterloo, Canada, and the Technical University of Munich, Germany, have proved theoretically that quantum computers can solve certain problems faster than classical computers. * We’re also seeing a lot of interest around human computer interaction - AR, VR, voice, neural interfaces. We had a presentation at QCon San Francisco from CTRL-labs, who are working on neural interfaces - in this case interpreting nerve signals - and they have working prototypes. Much like touch this could open up computing to another whole group of people.

On this week’s podcast, Wes Reisz talks with Brian Goetz. Brian is the Java Language Architect at Oracle. The two start with a discussion on what the six-month cadence has meant to the teams developing Java. Then move to a review of the features in Java 9 through 12. Finally, the two discuss the longer-term side projects (such as Amber, Loom, and Valhalla) and their role in the larger release process for the JDK. * The JVM’s sixth-month cadence changed the way the JDK is delivered and planned. While it definitely provides more rapid delivery at expected intervals, the release train approach turned out to also improve flexibility and efficiency. * Oracle JDK and OpenJDK are almost identical. Most of the JDK distributions are forks from OpenJDK with different bug fixes and backports applied. So the difference between the distributions now is largely which bug fixes are picked up. * Local Variable inference (which was released as part of Java 10) illustrated the tension on making changes to the language. Many people wanted the change, but many others felt it would enable people to write bad code. Oracle had to balance the two views when making the change. * The number of Java versions allow finer grain decision making on what is appropriate for an application. With the adoption of containers, applications are bundled with an exact JDK version rather than having to use one from a systems level. The different versions give developers more options. * Incubating features are new libraries added to the JDK. They were offered starting with Java 9 as a way for people to test and offer feedback more rapidly. With Java 12, preview features will be released. Preview features are similar but are core platform and language features. * Shenandoah and ZGC are both low latency garbage collectors. They originally came from different sources. While both garbage collectors are similar, each has different performance characteristics under different workloads. The two garbage collectors represent options available to JVM developers. * Most non-trivial JDK features take more than six months to develop. Longer term side projects like Amber, Loom, Valhalla are where these features are developed prior to being released with a version of the JDK. The projects range from language enhancements to concurrency work.

This week on the InfoQ Podcast, Wes Reisz talks to Tanya Reilly (Principal Engineer at Squarespace and previously a staff SRE at Google). Tanya discusses her research into how the fire code evolved in New York and draws on some of the parallels she sees in software. Along the way, she discusses what it means to be an SRE, what effective aspects of the role might look like, and her opinions on what we as an industry should be doing to prevent disasters. This podcast features discussion on paved roads, prevention, testing, firefighting (in software), and reliability questions to ask throughout the software lifecycle. Why listen to this podcast: - Teams increasingly are responsible for the entire software lifecycle. When this happens, they think about the software differently because they know their the ones that will get paged if it fails. This idea is at the core of the “You Build It, You Run It” philosophy in DevOps. - The role of SRE is to define how to do things in a really reliable way. The focus is to make the majority of the operations work go away, and, for the things that can’t go away, it’s as easy as possible. - At the very start of a project (when you’re writing the initial design), you should be thinking about the dependencies for a system and how will those that follow with be able to determine that. A great way to do this is to offer an API that people will want to use and then instrument it. - We can learn a lot from the growth of fire safety regulations as metaphors for software, including: fireproof interior walls, socializing best practices, software inspections, and circuit breakers are all examples. - The work SREs do varies in many places. SREs range from making recommendations on patterns to library creators in other areas. Occasionally, SREs are firefighters of last resort. In these cases, they’re the last resort though. - We use error budgets and SLOs to quantify how many much risk we’re comfortable taking. It’s used to inform how much less (or more risk) we’re willing to take on. - We need to consider software reliability throughout the full cycle of software development. When you build systems. Think about as if there will not be someone on call for it . You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq

On today’s podcast, Wes Reisz talks with Jason Maude of Starling Bank. Starling Bank is a relatively new startup in the United Kingdom working in the banking sector. The two discuss the architecture, technology choices, and design processes used at Starling. In addition, Maude goes into some of the realities of building in the cloud, working with regulators, and proven robustness with practices like chaos testing. Why listen to this podcast: - Starling Bank was created because the government lowered the barrier to entry for banking startups in reaction to previous industry bailouts. - The system is composed of around 19 applications hosted on AWS and running Java and backed by a PostgreSQL database. - These applications are not monolithic but are focused around common functionality (such as a Card or Payment Service). - Java was chosen primarily because of its maturity and long term viability/reliability in the market. - The heart of Starling is every action the system takes happens at least once and at most once. To help with these rules, everything in their system uses as a correlation id (UUID) and are used to make sure these two rules are met. You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq

Martin Fowler chats about the work he’s done over the last couple of years on the rewrite of the original Refactorings book. He discusses how this thought process has changed and how that’s affected the new edition of the book. In addition to discussing Refactors, Martin and Wes discuss his thoughts on evolutionary architecture, team structures, and how the idea of refactors can be applied in larger architecture contexts. Why listen to this podcast: - Refactoring is the idea of trying to identify the sequence of small steps that allows you to make a big change. That core idea hasn’t changed. - Several new refactorings in the book deal with the idea of transforming data structures into other data structures, Combine Functions into Transform for example. - Several of the refactorings were removed or not added to the book in favor of adding them to a web edition of the book. - A lot of the early refactorings are like cleaning the dirt off the glass of a window. You just need them to be able to see where the hell you are and then you can start looking at the broader ones. - Refactorings can be applied broadly to architecture evolution. Two recent posts How to break a Monolith into Microservices, by Zhamak Dehghani, and How to extract a data-rich service from a monolith by Praful Todkar on MartinFowler.com deal with this specifically. - Evolutionary architecture is a broad principle that architecture is constantly changing. While related to Microservices, it’s not Microservices by another name. You could evolve towards or away from Microservices for example. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2QbdHej You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2QbdHej