The InfoQ Podcast

In this podcast, Matthew Skelton and Manuel Pais, co-authors of the book Team Topologies, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the role of a modern software architect, how team design impacts software architecture, creating “team APIs” in order to reduce cognitive load, and the benefits of building a “thinnest viable platform”. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/3iyapiT Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

Today on The InfoQ Podcast, Wes Reisz talks to Pat Helland about the relationship between software architecture and urban planning. Pat explores planning for future growth, regulations/standards, and communication practices that cities--and software architecture--had to evolve to use. He uses these comparisons to distil lessons that architects can use in building distributed systems. A key theme throughout the podcast is constraints improve system design by restraining project scope. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/33IFR8A Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

In this podcast, John DesJardins, field CTO and VP solution architecture at Hazelcast, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how in-memory data grids have evolved, use cases at the edge (IoT, ML inference), integration of stream processing APIs and techniques, and how data grids can be used within application modernization. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/2FtBIgp Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

Charles Humble talks to Akhilesh Gupta, the technical lead for LinkedIn's real-time delivery infrastructure, and also LinkedIn messaging. They discuss the architecture behind LinkedIn’s real-time platform, its building blocks, the frameworks used and other technical details. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/2DujU3U Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

Today on the InfoQ Podcast, Yan Cui (a long time AWS Lambda user and consultant) and Wes Reisz discuss serverless architectures. The conversation starts by focusing on architectural patterns around choreography and orchestration. From there, the two move into updates on the current state of serverless cold start times, distributed tracing, and state. Today’s podcast, while not specific to AWS, does lean heavily on Yan’s expertise with AWS and AWS Lambda. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/2YPOLj5 Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

In this podcast, Liran Haimovitch, CTO at Rookout, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the concept of “understandability” and how this relates to building modern software systems, how complexity impacts a system’s understandability, and the benefits of live debugging tooling. Listen to the podcast for more. Curated transcript and more information on the podcast: https://bit.ly/3ggFE0r Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/

In this podcast, Ana Medina, senior chaos engineer at Gremlin, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how enterprise organisations are adopting chaos engineering with the requirements for guardrails and the need for “status checks” to ensure pre-experiment system health; how to run game days or IT fire drills when everyone is working remotely; and why teams should continually invest in learning from past incidents and preparing for inevitable failures within systems. Listen to the podcast for more. Curated transcript and more information on the podcast: https://bit.ly/2Pmsq7F Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/

In this podcast, Stefan Prodan, developer experience engineer at Weaveworks and creator of the Flagger project, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how progressive delivery extends the core ideas of continuous delivery; how the open source Flagger Kubernetes operator can be used to implement a progressive delivery strategy via canary releasing with an API gateway or service mesh; and the new “GitOps toolkit” that has evolved from the Flux continuous delivery operator. Listen to the podcast for more. Curated transcript and more information on the podcast: https://bit.ly/3f9CZ8p Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: https://www.infoq.com/software-architects-newsletter/

In this podcast, Shannon Williams, co-founder and president at Rancher Labs and Darren Shepherd, co-founder and CTO at Rancher Labs sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the adoption of hybrid cloud across organisations, the evolution of Kubernetes as a key abstraction for portability and cross-cloud security, running thousands of Kubernetes clusters at the edge, and the value of open standards. Why listen to this podcast - Organisations are adopting hybrid cloud strategies. The use of containers to package and run applications across clouds has seen large adoption over the past five years. Containers and Kubernetes are everywhere: the datacenter, the edge, embedded systems, and other locations. - Two enterprise use cases for Kubernetes stand out: providing standardised abstractions and APIs to increase portability across vendors and cloud platforms; and providing a framework and homogenised foundation on which to build and implement (cross cloud) security solutions. - Open standards support interoperability and drive innovation. The CNCF is becoming the natural home for open cloud technologies. The Rancher team have donated Longhorn, their cloud-native distributed storage platform for Kubernetes that was recently announced as generally available, to the CNCF. - With the success of lightweight Kubernetes distributions, such as Rancher’s K3s, engineers are starting to deploy standalone Kubernetes clusters “by the thousands” to edge locations. Rancher has recently released Fleet, a new open source project that is focused on managing large collections (“fleets”) of Kubernetes clusters. - Many developers and end users of Kubernetes simply want a platform-as-a-service (PaaS)-like experience. The next 12 months will see the community focus on the simplification of the Kubernetes ecosystem. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/31ZjYT1 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/31ZjYT1

In this podcast, Nora Jones, Co-Founder and CEO at Jeli and co-author of O’Reilly’s “Chaos Engineering: System Resiliency in Practice”, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: chaos engineering and resilience engineering, planning and running effective chaos experiments, and learning from incidents. Why listen to this podcast: - The chaos engineering and resilience engineering fields, although inextricably linked, are often incorrectly conflated. Resilience engineering is focused on “identifying and then enhancing the positive capabilities of people in organizations that allow them to adapt effectively and safely under varying circumstances.” - The UX of internal or engineering-focused tooling, such as chaos experimentation tooling, is extremely important. However, engineers that create these tools often overlook the value of UX, or don’t have the relevant skills in user design research to undertake this. - We all work in socio-technical systems. It is important to take the time to understand both aspects. Developing empathy and working alongside teams that you are trying to influence is essential. It is extremely important to continually work to build correct “mental models” of a system. - The before and after of running a chaos experiment is as important as running the experiment itself. However, the aspects of planning, creating effective hypotheses, and analysing and disseminating the results are often under-resourced. - Incident analysis can be a catalyst to help you understand more about your system. The Learning from Incidents website, alongside books such as Sidney Dekker’s The Field Guide to Understanding Human Error and Scott Snook’s Friendly Fire, can provide excellent background information to these topics. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2C4R6xL You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2C4R6xL

In this podcast, Rob Skillington, co-founder and CTO at Chronosphere, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: metrics collection at scale, multi-dimensional metrics and high-cardinality, developer experience with platform tooling, and open standards related to observability. Why listen to this podcast: - Over the past ten years the requirements related to monitoring and alerting, and the approach taken to implement this, has changed considerably. Compute is now ephemeral and dynamic, services are more numerous, and engineers want to instrument more things. Scalability of a monitoring solution is vitally important. - One of the challenges with metric data is the limited information for providing context for collected values. This can be solved by using multi-dimensional metrics. Dimensions of a metric are name-value pairs that carry additional data to describe the metric value. High dimensionality can lead to high cardinality. - Uber’s M3 metrics collection system initially used open source components such as Cassandra and ElasticSearch for storage and indexing. As the scale of usage of M3 increased, these OSS components were gradually replaced by custom components, such as M3DB. - Building an effective user experience for operational tooling, especially observability-foused tooling, is vitally important. Engineers will be interacting with these tools on a daily basis. They will also be relying on these tools for both alerting and being able to locate and understand what is occurring during production issues. - Open standards are vitally important for interoperability. The OpenMetrics project is an effort to create an open standard for transmitting metrics at scale, with support for both text representation and protocol buffers. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3i3dMPO You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3i3dMPO

In this podcast, Johnny Boursiquot, Site Reliability Engineer at Heroku, sat down with InfoQ podcast co-host Daniel Bryant and discussed topics that included: why Go is a useful language for building Function-as-a-Service (FaaS) style applications; how Heroku implement the role of Site Reliability Engineer (SRE); and why the ability to teach is such a valuable skill. Why listen to this podcast: - Go is a useful language for building Function-as-a-Service (FaaS) style applications. The ability to build Go applications into a static binary reduces the need for dependency management, and the quick runtime and application start time is good for initiation and scaling - The FaaS development toolchain has improved over the years. Many cloud providers now provide local runtimes, e.g. AWS SAM Local, and service simulators, e.g. LocalStack. Testing in production is facilitated by the ability to do dark launches and canary releasing at the ingress/API gateway - Developing “serverless” applications typically does not remove the need for operational expertise on a development team. Designing systems appropriately and getting the most out of the runtime (with minimal cost) requires knowledge of the underlying infrastructure components - The role of Site Reliability Engineering (SRE) looks different across practically every organisation. The Heroku SRE team have adapted well-established patterns and practices into their roles. They act as “diplomats”, working closely with product teams to share knowledge around operational best practices - The ability to teach is a valuable skill, regardless of your job. Teaching people to code or to embrace important operational principles is extremely rewarding. - Engineers who teach must seek to escape the pull of their ego; by focusing on the needs of the people you are teaching, much more progress can be made. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2UV0tqK You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2UV0tqK

In this podcast, Matt Debergalis, Founder and CTO at Apollo, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the motivations for GraphQL, the Apollo Data Graph platform, data modelling in an enterprise context, and how incrementally adopting GraphQL can help with decoupling the evolution of frontend and backend systems. Why listen to this podcast: - The challenges of defining client-side-friendly data models, building maintainable and composable backend APIs, and moving data from the cloud to a client application contributes to making modern software development difficult and time consuming. - GraphQL is an open-source data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data. GraphQL lets application developers describe the data they need and bring that data into the screens that they are building for their users. - The Apollo Data Graph platform is a middleware layer that provides a way of decoupling the core business APIs from the client-side consumption patterns. Apollo can implement cross-cutting concerns, such as transaction management, which mitigates the need to implement this in (potentially multiple) client-side applications. - Apollo makes it possible to build a “data graph”: a series of graphs that are composed from an organisation’s data for use within client-side applications. A data graph is especially valuable in larger enterprises because it is here that many (money making) systems with existing APIs need to be combined to meet new business requirements. - GraphQL can be adopted in an incremental fashion. To begin adoption just build the simplest possible graph that matches the needs of the first application, the first screen, or the first component that is required to transition over to the graph. Then let that graph evolve. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3fhaeqP You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3fhaeqP

In this podcast, Lin Sun, senior technical staff member and master inventor at IBM, and Neeraj Poddar, engineering lead and architect at Aspen Mesh, sat down with InfoQ co-host Daniel Bryant. Topics discussed included: the evolution of service mesh data planes and control planes, the new Istio 1.5 architectures, Istio WebAssembly extension support, and the future of service mesh technology. Why listen to this podcast: - A service mesh in one implementation approach to provide service discovery, traffic management, and cross-cutting communication concerns that engineers see when they adopt (micro)service-based. - The data plane of most modern service mesh implementations run out-of-process as a proxy sidecar. This has evolved from library based implementations, such as Airbnb’s SmartStack or Netflix’s OSS libraries. - The recent release of Istio 1.5 saw the deployment packaging of the control plane move from a microservice-based approach to that of a monolithic implementation, named “istiod”. - Istio now also supports data plane extensions written in WebAssembly (Wasm). These extensions can modify requests and responses and perform out-of-band actions, such as authentication and authorization. - Standardisations like the Service Mesh Interface (SMI) can add a lot of value, but the user requirements, common use cases, and the core abstractions of the underlying technology must be well understood. - Multi-cluster and mesh expansion (out-of-cluster) support is continually improving in Istio and many other service mesh implementations.

Today on the InfoQ Podcast, Wes Reisz talks with one of the thought leaders in Microservices, CI/CD, and Cloud -- Sam Newman. The podcast covers many of the topics, techniques, and patterns that Sam writes about in his latest book, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith. Topics covered in the podcast include understanding the problem you’re trying to solve, organizational/people changes when it comes to microservice architectures, database strategies for decomposing monolithic datastores, and why we’re seeing projects reverting from microservices to monoliths. Why listen to this podcast: - Fundamentally, microservices are distributed systems. Distributed systems have baggage (complexity) that comes along with them. The best way to deal with this complexity is not to address it. Try to solve the problem in other ways before choosing to take an organization to microservices. - A common issue that large enterprises run into that might be a strong indicator for implementing microservices occurs when lots of developers are working on a given problem and they’re getting in each other’s way. - A useful structure to follow with microservices is to make sure each service is owned by exactly one team. One team can own more than one service but having clear ownership of who owns a service helps in some of the operational challenges with microservices. - A release train should be a stop in the journey towards continuous delivery. It’s not the destination. If you find that you can only release in a release train, you are likely building a distributed monolith. - There are challenges of operating microservices when the end customer has to operate and manage it. These challenges are part of why we’re seeing projects move from microservices to process monoliths. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2XvGzmF You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2XvGzmF

In this podcast, Tracy Miranda sat down with InfoQ podcast co-host Daniel Bryant. Miranda, Director of Open Source Community at CloudBees, and board chair at the Continuous Delivery Foundation (CDF), discussed topics that included: the aims of the CDF and an outline of the current hosted projects, the need for open standards and interoperability in the CD space, and the benefits offered by progressive delivery and software supply chain management. Why listen to this podcast: - The Continuous Delivery Foundation (CDF) serves as the vendor-neutral home of many projects within continuous delivery space, including: Jenkins, Jenkins X, Spinnaker, Tekton, and Screwdriver.cd - Jenkins X is a Kubernetes-native continuous delivery solution for cloud applications. This project uses a completely new architecture and code base in comparison with the original Jenkins project. - Spinnaker is an open source, multi-cloud continuous delivery platform. The Tekton Pipelines project provides Kubernetes-style custom resources for declaring continuous integration and delivery pipelines. Spinnaker can use Tekton as its pipeline engine. - In addition to providing a neutral home for projects within the CD space, the CDF is also aiming to help define appropriate terminology, open standards, and abstractions. This will assist with interoperability between CD components, and also promote innovation in the areas that can provide the most value. - The CDF is also aiming to facilitate software testing, progressive delivery, and software supply chain management. Wide ranging topics such as observability and security are important will play an important role here. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bzoYiz You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3bzoYiz

In this podcast, Marty Abbott and Tanya Cordrey sat down with InfoQ podcast co-host Daniel Bryant. Abbott, CEO and co-founder of AKF Partners, and Cordrey, partner at AKF Partners, discussed topics that included: their learning from working together in the early days of eBay, why and how to avoid creating software systems that are composed of deep call chains of microservices, and how to build effective product teams. Why listen to this podcast: - First introduced in the book “The Art of Scalability”, the AKF Scale Cube is a model for segmenting software components, defining microservices, and scaling products. It also creates a common language for teams to discuss scale related options in designing solutions. - The microservice architectural pattern is best used for implementing the “breadth” of business functionality. Engineers should avoid building deep call chains of services, as this can increase the probability of failure, and can also increase the challenges of locating and diagnosing issues. Code libraries can often be used more effectively to implement “depth” within services. - The AKF Availability Cube is a new model to guide discussions on how to achieve high availability. This model can also be used as a mathematical tool to evaluate the theoretical “as designed” availability of existing systems. - Building products using cross-functional teams is an effective approach. However, care should be taken not to accidentally create unnecessarily large teams, as this can add communication and coordination friction to the delivery process. - Teams should make a conscious choice to adopt new technologies, and understand the benefits and tradeoffs with doing so. Managing risk, and in particular, technology lifespan risk, is an important part of the value engineers provide to the business. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2AbpZjT You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2AbpZjT

In this podcast, Daniel Bryant sat down with Dave Sudia, senior DevOps engineer at GoSpotCheck. Topics discussed included: the benefits of PaaS; building a platform with Kubernetes as the foundation; selecting open source components and open standards in order to facilitate the evolution of a platform; and why care should be taken to prioritize the developer experience and create self-service operation of the platform. Why listen to this podcast: - When starting a business and searching for product-market fit, creating an application using a monolithic code base deployed onto a commercial Platform as a Service (PaaS) product is a very effective way of iterating fast and minimising operational costs. - There may come a point where the PaaS cannot provide bespoke requirements, or it has trouble scaling, or the costs become prohibitive. At this point many teams choose to build a custom platform using cloud technologies, such as Kubernetes. - Building a Kubernetes platform can be an effective solution, but appropriate effort needs to be put into designing, building, and maintaining the platform. The platform effectively becomes another product within the business that must be managed accordingly. - Embracing open standards provides many benefits, especially for the long term. Implementations that are consumed through well-defined interfaces and abstraction can be more readily swapped at a later point in time. It is also generally easier to integrate components that share common interfaces. - Attention and resources must be provided to create an effective developer experience for the platform. It is essential to prioritize self-service operations, and also to understand the core requirements of the engineers and QA specialists that will be using the platform during their daily work. - Establishing an effective continuous delivery pipeline can enable more repeatable and scalable testing of applications, and also allows the codification of cross-functional requirements. - The cloud native landscape has now evolved to a point where most of the frameworks and tooling required to build a platform have become viable for general purpose usage. However, some assembly may still be required, and engineers should be prepared for change, as the ecosystem moves fast. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bj3BTp You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3bj3BTp

Today on The InfoQ Podcast, Wes Reisz talks with Peter Bourgon. Peter is a distributed system engineer working on Fastly. His area of interest is around coordination free replicated systems. The two engineers talk about the space of Conflict-Free Replicated Data Types (CRDTs) specifically in the context of edge compute. Topics covered on the podcast include Edge compute, CRDTs, CAP Theorem, and challenges around building distributed systems. Why listen to this podcast: - CRDTs (conflict-free replicated data types) are a class of coordination free replication systems (or systems that don’t require strategies such as leader election). - An easy way to think of a CRDT is as an associative, commutative, and idempotent data structure plus the operations needed to use it. The edge is an overloaded term that people tend to define based on where they sit on a spectrum between the customer and the data center. Fastly’s edge is away from the data center and but not to the telephone pole or handset. - RAFT and Gossip are two alternative approaches to using a coordination free replication system like CRDTs. To get the properties of a CRDT and have useful data types, you have to pay a cost in size and often bytes on the wire. These are challenges that continue to need solutions. - Modern Distributed systems and data structures like CRDTs require you to start thinking about state in the system itself. It’s not unusual for a system today to give you multiple results back that the system will have to handle or merge. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dTZOO4 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3dTZOO4

In this podcast, Daniel Bryant sat down with Joe Duffy, founder and CEO at Pulumi, and discussed several infrastructure-themed topics: the evolution of infrastructure as code (IaC), the way in which the open source Pulumi framework allows engineers to write IaC using general purpose programming languages such as JavaScript and Go, and the future of multi-cloud environments. Why listen to this podcast: ● Infrastructure as Code (IaC) enables engineers to programmatically define the configuration and provisioning of computing infrastructure, on-premises hardware, and cloud services. ● Traditional IaC tools were often imperative, requiring engineers to define and enumerate the necessary steps and SDK calls in order to configure the underlying infrastructure. ● Modern IaC tools like HashiCorp’s Terraform, AWS CloudFormation and other related cloud vendor tooling enable engineers to write declarative code to define a required state of the infrastructure. The tools parse the declarative configuration and take appropriate action to enact the specified state, for example, calling SDKs and APIs, verifying results, iterating etc. ● Pulumi is an open source framework that enables engineers to define IaC using general purpose programming languages, such as Node, Python, .NET Core, and Go. ● Pulumi allows imperative specification of IaC. Engineers can use their favourite language-specific features, idioms, and patterns. The use of language modules, packages, and libraries can also enable code reuse. ● Under the hood, Pulumi transforms code written in the supported languages to a declarative specification model. This model is then used to enact the required infrastructure state. ● Frameworks like Pulumi enable engineers to deploy and configure infrastructure across multiple cloud vendors and services (including Kubernetes clusters). More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3dyxVee You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3dyxVee

In this podcast Charles Humble spoke to Dylan Schiemann, co-creator of Dojo and InfoQ’s JavaScript and Web Development lead editor, about the history and current state of Dojo, and key emerging trends in the JavaScript landscape today. Key topics include Dojo’s adoption of Typescript, web components, and client-side libraries such as Svelte and Stencil. Why listen to this podcast: - Modern Dojo (2.0 and upwards) is focussed on being a very small, opinionated reactive framework, but with a lot of the components you need to build a modern JavaScript application built in. - The framework tries to align closely to standards, for example using Web Components extensively for UI components, alongside ES modules and promises. The use of standards, as well as the convergence towards the reactive programming model for web UI, has improved interoperability, though there are some limitations such as the lack of an easy way to share resources across web components. - Dojo was one of the first frameworks to make the decision to switch to Typescript, though it took some time to make that transition. The switch was mainly motivated by TypeScript’s support for interfaces, but it wasn’t until Typescript 2.6 they felt able to ship Dojo 2. - On the client side we’re paying close attention to Svelte and Stencil as two particularly interesting client-side frameworks. - We’ve moved Web Components from early adopter to early majority on the trend report, based on the fact that all browsers accept IE now natively support it, but also large companies such as Apple, Nike and ESPN are deploying web components and their sites. Apple’s iTunes implementation, for example, now uses web components. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Qy75Jr You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Qy75Jr

In this podcast, Daniel Bryant sat down with Gareth Rushgrove, Director of Product Management at Snyk. Topics covered included Kubernetes as a platform, application abstractions, continuous delivery, and implementing good security practices in the cloud native space. Why listen to this podcast: - The value provided by Kubernetes depends on an organisation’s context. Kubernetes acts as both a series of lower-level building blocks for a platform, and also as a very powerful API for deploying and operating container-based applications. - Kubernetes provides several useful abstractions for engineers. For example, Pods, Deployments, and Services. However, Kubernetes doesn’t have an “application”-focused abstraction. Tools such as Helm and specifications like the Cloud Native Application Bundle (CNAB) are driving innovation in this space. - There is a large amount of open source Kubernetes tooling. This has been created by a range of vendors, groups, and individuals. Encouraging this diverse mix of participation is beneficial for the long-term health of the ecosystem. - The Cloud Native Computing Foundation (CNCF) provides a space for people to collaborate regardless of their current organisational affiliations. - Defining appropriate standards within the cloud native space is useful for enabling interoperability and providing common foundations for others to innovate on top of. - Security challenges within IT are socio-technical. Security teams working with cloud native technologies will benefit from continual learning, developing new skills, and researching new tools. For example, the defaults of Kubernetes aren’t necessarily secure, but this can be readily addressed with appropriate configuration. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/38PLPFb You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/38PLPFb

- A Micro frontends is an approach to developing frontends that attempts to take some of the same benefits from Microservices and apply them to frontend development. - Microfront ends can be developed with different technologies and ownership of components on a single view. However, DAZN took a vertical approach to build them. Each Micro frontend is loaded into an app shell that offers an API for crosscutting concerns. Only one Micro frontend is loaded at a time into the app shell. - The ‘Inverse Conway Maneuver’ recommends evolving your team and organizational structure to create the architecture you want. - DAZN derisks deployments by using canaries implemented with Lambda at the Edge on Cloudfront. For code deployments, each of the Micro frontends can be introduced with a limited scope and then expanded once proven stable. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/38BQAC0 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/38BQAC0

In this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows. Why listen to this podcast: - Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage. - There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized. - The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh. Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way. - Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers. The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/39exTWl You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/39exTWl

In this podcast, Daniel Bryant sat down with Brittany Postnikoff, a computer systems analyst specialising on the topics of robotics, embedded systems, and human-robot interaction. Topics discussed included: the rise of robotics and human-robot interaction within modern life, the security and privacy risks of robots used within this context, and the potential for robots to be used to socially engineer people. Why listen to this podcast: - Physical robots are becoming increasingly common in everyday life, for example, offering directions in airports, cleaning the floor in peoples’ homes, and acting as toys for children. - People often imbue these robots with human qualities, and they trust the authority granted to a robot. - Social engineering can involve the psychological manipulation of people into performing actions or divulging confidential information. This can be stereotyped by the traditional “con”. - As people are interacting with robots in a more human-like way, this can mean that robots can be used for social engineering. - A key takeaway for creators of robots and the associated software is the need to develop a deeper awareness of security and privacy issues. - Software included within robots should be patched to the latest version, and any data that is being stored or transmitted should be encrypted. - Creators should also take care when thinking about the human-robot UX, and explore the potential for unintended consequences if the robot is co-opted into doing bad things. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2v5QTav You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2v5QTav

In this podcast, Daniel Bryant sat down with Anurag Goel, Founder and CEO of Render. Topics covered included: the evolution of cloud platforms; simplifying developer experience; running large-scale workloads on Kubernetes; and the future of tooling and platforms within the cloud native computing space. Why listen to this podcast: - Render is aiming to be the next generation of cloud provider. Developers deploy and manage applications via a Platform-as-a-Service (PaaS) -like experience using custom simplified YAML configuration. - Render is built on top of Kubernetes, but the internals and configuration of this orchestration framework is not exposed to end user developers. - Many large scale usages of traditional cloud vendor platforms require the formation of specialised in-house “DevOps” teams. The provision of virtualisation and API-driven operation via the cloud providers was revolutionary, but it didn’t fundamentally change the existing platform paradigm. - Arguably platform usability may have taken a step back with the arrival of public cloud vendor platforms. For example, developers may just want to write code, and not have to write complicated deployment descriptors. Operations team may want to focus on supporting engineers and advising on performance and scale, rather maintaining cloud provisioning scripts. - The Render team are planning to run all future workloads of self-managed Kubernetes, rather than use a hosted offering, due to them experiencing implementation bugs when running their clusters at medium-to-large scale. - The Cloud Native Computing Foundation (CNCF) is encouraging large amounts of innovation within the cloud platform space. However, due to the Cambrian explosion of the Cloud Native Landscape over the past several years, there must surely be consolidation of tools, platforms, and vendors in the near future. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2H2lY1s You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2H2lY1s

In this podcast, Daniel Bryant sat down with Greg Law, CTO at Undo. Topics discussed included: the challenges with debugging modern software systems, the need for “hyper-observability” and the benefit of being able to record and replay exact application execution; and the challenges with implementing the capture of nondeterministic system data in Undo’s LiveRecorder product for JVM-based languages that are Just-In-Time (JIT) compiled. Why listen to this podcast: - Understanding modern software systems can be very challenging, especially when the system is not doing what is expected. When debugging an issue, being able to observe a system and look at logging output is valuable, but it doesn’t always provide all of the information a developer needs. Instead we may need “hyper observability”; the ability to “zoom into” bugs and replay an exact execution. - Being able to record all nondeterministic stimuli to an application -- such as user input, network traffic, interprocess signals, and threading operations -- allows for the replay of an exact execution of an application for debugging purposes. Execution can be paused, rewound, and replayed, and additional logging data can be added ad hoc. - Undo’s LiveRecorder allows for the capture of this nondeterministic data, and this can be exported and shared among development teams. The UndoDB debugger, which is based on the GNU Project Debugger, supports the loading of this data and the execution and debugging in forwards and reverse execution of the application. There is also support for other debuggers, such as that included within IntelliJ IDEA. - Advanced techniques like multi-process correlation reveal the order in which processes and threads alter data structures in shared memory, and thread fuzzing randomizes thread execution to reveal race conditions and other multi-threading defects. - The challenges of using this type of technology when debugging (micro)service-based application lies within the user experience i.e. how should the multiple process debugging experience be presented to a developer? Live Recorder currently supports C/C++, Go, Rust, Ada applications on Linux x86 and x86_64, with Java support available in alpha. Supporting the capture and replay of data associated with JVM language execution, which contain extra abstractions and are often Just-In-Time (JIT) compiled, presented extra challenges. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37XLUa0 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37XLUa0

Today on The InfoQ Podcast, Wes Reisz speaks with CEO and founder of Solo Idit Levine. The two discuss the Three Pillars of Solo around Gloo, their API gateway, interoperability of service meshes (including the work on Service Mesh Interface), and on extending Envoy with Web Assembly (and the recently announced Web Assembly Hub). Why listen to this podcast: - Gloo is a Kubernetes-native ingress controller and API gateway. It’s built on top of Envoy and at its core is open source. - The Service Mesh Interface (SMI) is a specification for service meshes that runs on Kubernetes. It defines a common standard that can be implemented by a variety of providers. The idea of SMI is it’s an abstraction on top of service meshes, so that you can use one language to configure them all. - Autopilot is an open-source Kubernetes operator that allows developers to extend a service mesh control plane. - Lua has been commonly used to extend the service mesh data plane. Led by Google and the Envoy community, web assembly is becoming the preferred way of extending the data plane. Web assembly allows you to write Envoy extensions in any language while still being sandboxed and performant. - WebAssembly Hub is a service for building, deploying, sharing, and discovering Wasm extensions for Envoy. - Wasme is a docker like an open-source commandline tool from Solo to simplify the building, pushing, pulling, and deploying Envoy Web Assembly Filters. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37sYIoE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37sYIoE

Today, on The InfoQ Podcast, Wes Reisz talks with Gunnar Morling. Gunnar is a software engineer at RedHat and leads the Debezium project. Debezium is an open-source distributed platform for change data capture (CDC). On the show, the two discuss the project and many of its use cases. Additionally, topics covered on the podcast include bootstrapping, configuration, challenges, debugging, and operational modes. The show wraps with long term strategic goals for the project. Why listen to this podcast: - CDC is a set of software design patterns used to react to changing data in a data store. Used for things like internal changelogs, integrations, replication, and event streaming, CDC can be implemented leveraging queries or against the DB transaction log. Debezium leverages the transaction log to implement CDC and is extremely performant. - Debezium has mature source and sink connectors for MySQL, SQL Server, and MongoDB. In addition, there are Incumbating connectors for Cassandra, Oracle, and DB2. Community sink connectors have been created for ElasticSearch. - In a standard deployment, Debezium leverages a Kafka cluster by deploying connectors into Kafka Connect. The connectors establish a connection to the source database and then write changes to a Kafka topic. - Debezium can be run in embedded mode. Embedded mode imports Java library into your own project and leverages callbacks for change events. The library approach allows Debezium implementations against other tools like AWS Kinesis or Azure's Event Hub. Going forward, there are plans to make a ready-made Debezium runtime. - Out of the box, Debezium has a one-to-one mapping between tables and Kafka topic queues. The default approach exposes the internal table structure to the outside. One approach to address exposing DB internals is to leverage the Outbox Pattern. The Outbox Pattern uses a separate outbox table as a source. Inserts into your normal business logic tables also make writes to the outbox. Change events are then published to Kafka from the outbox source table. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3737GZB You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3737GZB

In this podcast, Daniel Bryant sat down with Kelsey Hightower, Staff Developer Advocate at Google. Topics covered included: the extensibility of Kubernetes, and why it has become the platform that other platforms are being built on top of; creating event-driven architectures and deploying these onto Function-as-a-Service (FaaS) platforms like the Kubernetes-based Knative and Google Cloud Run; and the benefits of learning, sharing knowledge, and building communities. Why listen to this podcast: - Kubernetes is a platform for building platforms. It may not be as opinionated as traditional Platform-as-a-Service (PaaS) offerings, but it has become popular due to its extensibility. There are PaaS-like solutions built on top of Kubernetes, such as OpenShift, Knative, and Cloud Run. - The creation of common interfaces within Kubernetes -- such as Custom Resource Definitions (CRDs), Container Networking Interface (CNI), and Container Runtime Interface (CRI) -- enabled the adoption of the platform by vendors and the open source community without everyone needing to agree on exactly how to implement extensions. - Although not every workload can be effectively implemented using an event-driven architecture, for those that can the Kubernetes-based Function-as-a-Service (FaaS) platforms like Knative and Cloud Run can handle a lot of the operational management tasks for developers. - Engineers may be able to get ~90% of the “service mesh” traffic management functionality they need from using a simple proxy. However, the separation of the control and data planes within modern service meshes, in combination with the rise in popularity of the sidecar deployment model, has provided many benefits within Kubernetes. - A lot of learning within software development and information technology is transferable. If you spend time going deep in a technology when you begin your career, much of what you learn will be useful when you come to learn the next technology. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/30alHC1 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/30alHC1

In this podcast, Daniel Bryant sat down with Katie Gamanji, Cloud Platform Engineer at Condé Nast International. Topics covered included: exploring the architecture of the Condé Nast Kubernetes-based platform; the importance of enabling self-service deployment for developers; and how the Kubernetes’ Federation API and Cluster API may enable more opportunities for platform automation. - Founded in the early 1900s, Condé Nast is a global media company that has recently migrated their application deployment platforms from individually-curated geographically-based platforms, to a standardised distributed platform based on Kubernetes and AWS. - The Condé Nast engineering team create and manage their own Kubernetes clusters, currently using CoreOS’s/Red Hat’s Tectonic tool. Self-service deployment of applications is managed via Helm Charts. - The platform team works closely with their “customer” developer teams in order to ensure their requirements are being met. - The Kubernetes Federation API makes it easy to orchestrate the deployment of applications to multiple clusters. This works well for cookie-cutter style deployments that only require small configuration differences, such as scaling the number of running applications based on geographic traffic patterns. - The Cluster API is a Kubernetes project to bring declarative APIs to cluster creation, configuration, and management. This enables more effective automation for cluster lifecycle management, and may provide more opportunities for multi-cloud Kubernetes use. - The Condé Nast platform Kubernetes Ingress is handled by Traefik, due to the good Helm support and cloud integration (for example, AWS Route 53 and IAM rule synchronization). The platform team is exploring the use of service mesh for 2020. - Abstractions, interfaces, and security will be interesting focal points for improvement in the Kubernetes ecosystem in 2020. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2FeYPrE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2FeYPrE

In this podcast, Daniel Bryant spoke to Joseph Jacks, Founder of OSS Capital and the Open Core Summit, and discussed topics including the open source and open core models, innovations within open source hardware and the RISC-V instruction set architecture, and current opportunities for disruption using commercial open source software. Why listen to this podcast: - Recently, open source software and the open core business model have driven a lot of innovation and created a lot of value, particularly within the cloud “as-a-service” space. - There has been some disagreement between the open source and commercially-focused communities, for example, in relation to the licencing models and how value is captured. - The Open Core Summit (OCS) is a new conference focusing on the intersection of commercialisation and open source software that aims to facilitate discussion in this space. - Organisations building around open source software can potentially look at large cloud vendors as partners. Public clouds can provide effective distribution, and typically focus on offering breadth of services rather than the depth of expertise that can be provided by a specialist company. - RISC-V is an open-source hardware instruction set architecture (ISA) based on the well-established reduced instruction set computer (RISC) principles. Leveraging RISC-V can reduce the time and cost of customising chip designs. - A lot of recent open source innovation has focused on the infrastructure layer within computing systems. This means that the application layer is now potentially ripe for disruption via commercial open source software. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2rDfYYU You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2rDfYYU

In this special year-end wrap-up podcast Wes Reisz, Shane Hastie, Daniel Bryant, and Charles Humble discuss what we’ve seen in 2019 and speculate a little on what we hope to see in 2020. Topics include business agility and Teal, what it means to be an ethical engineer, bringing your whole self to work, highlights from QCon and InfoQ during 2019, the rise of Python, and progress in quantum computing. Why listen to this podcast: * Business agility is one of the major themes that the InfoQ team has seen emerge this year, with stronger emphasis on outcomes over outputs. We’ve also seen a growing interest in ethics and the ethical implications of the work we all do. * On the programming languages front the rise of Python continues, driven largely by its popularity in data science. * As Kubernetes cements its dominant position we’re hoping to see a simplification of the workflows associated with it, as well as in areas like observability. * There have been several big announcements in quantum computing in the past year, and this is an area we continue to watch with interest. * Another key trend for next year is edge computing. The edge of the cloud infrastructure has an amazing amount of available compute resource, as does the device edge. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Z0Q9OI You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Z0Q9OI

Josh Wills, a software engineer working on data engineering problems at Slack, discusses the Slack data architecture and how they build and observe their pipelines. Josh, along with color commentary such as the move from IC to manager (and back), discusses recommendations, tips, tools, and lessons Slack engineering teams discovered while building products like Slack Search. The podcast covers machine learning, observability, data engineering, and general practices for building highly resilient software. Why listen to this podcast: - Slack has a philosophy of building only what they need. They have a don’t reinvent the wheel mindset. - Slack was originally a PHP monolith. Today, it is largely Hack-lang, HHVM, and several Java and Go binarys. On the data side, application logs are in Thrift (there is a plan to migrate to protobuf). Events are processed through a Kafka cluster that handles 100,000s of events per second. Everything is kept in S3 with a large Hive metastore. EMR is spun up on demand. Presto, Airflow, Slack, Snowflake (business analytics), Quiver (key value store) are all used. - ML worked best for Slack when it was used to help people answer questions. Things like Learn to Rank (LTR) become the most effective use of ML for Slack. - You can get pretty far with rules. Use machine learning when that’s all that’s left. - When you start applying observability to your data pipeline, a key lesson for Slack was to really focus on structured data, tracing, high cardinality events. This let them really use the tools they were already familiar with (ELK, Prometheus, Grafana) and go deep into understanding what’s happening in the systems. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PsVA4q You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2PsVA4q

In this podcast, Daniel Bryant sat down with Bryan Liles, senior staff engineer at VMware. Topics covered included: the challenges with deploying applications into Kubernetes, using the open source Octant tool to increase a user’s understanding of Kubernetes clusters, and how “serverless” technologies may influence the future approaches to building software. Why listen to this podcast: - Octant is a highly extensible platform for developers to better understand the complexity of Kubernetes clusters. Octant runs locally, using the local Kubernetes credentials. It currently displays information about a Kubernetes cluster and related applications as a web page. Soon this tool and resulting display will be provided as a standalone application. - The goal of Octant is to enable users to discover what they need to discover. The tool aims to provide context relevant to where a user is and what they are trying to achieve. The Octant plugin system allows integration with other tooling, such as logging and metrics frameworks. This aims to facilitate quick problem detection and resolution. - Cloud native platforms like Kubernetes are complicated, as there are lots of moving parts. The most important challenge to be tackled to increase the adoption of platforms like Kubernetes is “how do we move code from our IDEs to wherever it needs to run with the least amount of friction?”. Testing needs to be implicit, as does security verification, and the acts of deployment. Kubernetes needs its “Ruby on Rails” moment. - Creating “serverless” systems is an interesting approach, but we may currently be using this technology in a non-optimal way. For example, creating web applications using this technology enables scalability, but can lead to the creation of difficult to understand systems that also require a lot of boilerplate configuration. Arguably, a more interesting use case is implementing large-scale batch processing using simple event-driven models. - The Cloud Native Computing Foundation (CNCF) has created a series of communities of practice called Special Interest Groups (SIGs), such as SIG App Delivery. This allows folks with similar interests to work together as a community, focusing on solving a specific set of well-scoped problems. There are many ways to get involved, from discussions, to coding and creating documentation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/37iUwIG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/37iUwIG

Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. On today’s podcast, Wes and Victor talk about the realities of building machine learning in the browser. The two discuss the capabilities, limitations, process, and realities around using TensorFlow.js. The two wrap discussing techniques like Model distillation that may enable machine learning models to be deployed in smaller footprints like serverless. - While there are limitations in running machine learning processes in a resource constrained environment like the browser, there are tools like TensorFlow.js that make it worthwhile. One powerful use case is the ability to protect the privacy of a user base while still making recommendations. TensorFlow.js takes advantage of the WebGL library for its more computational intense operations. - TensorFlow.js enables workflows for training and scoring models (doing inference) purely online, by importing a model built offline with more tradition Python tools, and a hybrid approach that builds offline and finetunes online. To build an offline model, you can build a model with TensorFlow Python (perhaps using a GPU cluster). The model can be exported into the TensorFlow SaveModel Format (or the Keras Model Format) and then converted with TensorFlow.js into the TensorFlow Web Model Format. At that point, the can be directly imported into your JavaScript. - TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models and was made available by the Google AI team. It can give developers a quick jumpstart into using trained models. - Model compression promises to make models small enough to run in places we couldn’t run models before. Model distillation is a process where a smaller model is trained to replicate the behavior of a larger one. In one case, BERT (a library almost 500MB in size) was distilled to about 7MB (almost 60x compression). More on this: Quick scan our curated show notes on InfoQ https://bit.ly/32rWnab You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/32rWnab

In this podcast, Daniel Bryant spoke to Michelle Krejci, service engineer lead at Pantheon, about the Drupal and Wordpress webops-based company’s move to a microservices architecture. Michelle is a well-known conference speaker in the space of technical leadership and continuous integration, and she shared her lessons learned over the past four years of the migration. Why listen to this podcast: - The backend for the Pantheon webops platform began as a Python-based monolith with a Cassandra data store. This architecture choice initially enabled rapid feature development as the company searched for product/market fit. However, as the company found success and began scaling their engineering teams, the ability to add new functionality rapidly to the monolith became challenging. - Conceptual debt and technical debt greatly impact the ability to add new features to an application. Moving to microservices does not eliminate either of these forms of debt, but use of this architectural pattern can make it easier to identify and manage the debt, for example by creating well-defined APIs and boundaries between modules. - Technical debt -- and the associated engineering toil -- is real debt, with a dollar value, and should be tracked and made visible to everyone. Establishing “quick wins” during the early stages of the migration towards microservices was essential. Building new business-focused services using asynchronous “fire and forget” event-driven integrations with the monolith helped greatly with this goal. - Using containers and Kubernetes provided the foundations for rapidly deploying, releasing, and rolling back new versions of a service. Running multiple Kubernetes namespaces also allowed engineers to clone the production namespace and environment (without data) and perform development and testing within an individually owned sandboxed namespace. - Using the Apollo GraphQL platform allowed schema-first development. Frontend and backend teams collaborated on creating a GraphQL schema, and then individually built their respective services using this as a contract. Using GraphQL also allowed easy mocking during development. Creating backward compatible schema allowed the deployment and release of functionality to be decoupled.

In today’s podcast we sit down with Ryan Kitchens, a senior site reliability engineer and member of the CORE team at Netflix. This team is responsible for the entire lifecycle of incident management at Netflix, from incident response to memorialising an issue. Why listen to this podcast: - Top level metrics can be used as a proxy for user experience, and can be used to determine that issue should be alerted on an investigated. For example, at Netflix if the customer playback initiation “streams per second” metric declines rapidly, this may be an indication that something has broken. - Focusing on how things go right can provide valuable insight into the resilience within your system e.g. what are people doing everyday that helps us overcome incidents. Finding sources of resilience is somewhat “the story of the incident you didn’t have”. - When conducting an incident postmortem, simply reconstructing an incident is often not sufficient to determine what needs to be fixed; there is no root cause with complex socio-technical systems as found at Netflix and most modern web-based organisations. Instead, teams must dig a little deeper, and look for what went well, what contributed to the problem, and where are the recurring patterns. - Resilience engineering is a multidisciplinary field that was established in the early 2000s, and the associated community that has emerged is both academic and deeply practical. Although much resilience engineering focuses on domains such as aviation, surgery and military agencies, there is much overlap with the domain of software engineering. - Make sure that support staff within an organisation have a feedback loop into the product team, as these people providing support often know where all of the hidden problems are, the nuances of the systems, and the workarounds. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2LLwk8T You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2LLwk8T

In this podcast we sit down with Oliver Gould, co-founder and CTO of Buoyant. Oliver has a strong background in networking, architecture and observability, and worked on solving associated technical challenges at both Yahoo! and Twitter. Oliver is a regular presenter at cloud and infrastructure conferences, and alongside his co-founder William Morgan, you can often find them in the hallway track, waxing lyrical about service mesh -- a term they practically coined -- and trying to bring others along on the journey. Service mesh technology is still young, and the ecosystem is still very much a work in progress, but there have been several recent interesting developments within this space. One of these was the announcement of the service mesh interface (SMI) at the recent KubeCon EU in Barcelona. The SMI spec seeks to unlock service mesh integrators and implementers, as this can provide an abstraction that removes the need to bet on any single service mesh implementation. This can be good for both tool makers and enterprise early adopters. Many organisations like Microsoft and HashiCorp are involved with working alongside the community to help define the SMI, including Buoyant. In this podcast we summarise the evolution of the service mesh concept, with a focus on the three pillars: visibility, security, and reliability. We explore the new traffic “tap” feature within Linkerd that allows near real time in-situ querying of metrics, and discuss how to implement network security by leveraging the primitives like Service Account provided by Kubernetes. We also discuss how reliability features, such as retries, time outs, and circuit-breakers are becoming table stakes for infrastructure platforms. We also cover the evolution of the service mesh interface, explore how service meses may impact development and platforms in the future, and briefly discuss some of the benefits offered by the Rust language in relation to building a data plane for Linkerd. We conclude the podcast with a discussion of the importance of community building. Why listen to this podcast: - A well-implemented service mesh can make a distributed software system more observable. Linkerd 2.0 supports both the emitting of mesh telemetry for offline analysis, and also the ability to “tap” communications and make queries dynamically against the data. The Linkerd UI currently makes use the tap functionality. - Linkerd aims to make the implementation of secure service-to-service communication easy, and it does this by leveraging existing Kubernetes primitives. For example, Service Accounts are used to bootstrap the notion of identity, which in turn is used as a basis for Linkerd’s mTLS implementation. - Offering reliability is “table stakes” for any service mesh. A service mesh should make it easy for platform owners to offer fundamental service-to-service communication reliability to application owners. - The future of software development platforms may move (back) to more PaaS-like offerings. Kubernetes-based function as a service (FaaS) frameworks like OpenFaaS and Knative are providing interesting features in this space. A service mesh may provide some of the glue for this type of platform. - Working on the service mesh interface (SMI) specification allowed the Buoyant team to sit down with other community members like HashiCorp and Microsoft, and share ideas and identify commonality between existing service mesh implementations. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2m5DSJ6 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2m5DSJ6

Today on the podcast, Bernd Rucker of Camunda talks about event sourcing. In particular, Wes and Bernd discuss thoughts around scalability, events, commands, consensus, and the orchestration engines Camunda implemented. This podcast is a primer on considerations between an RDBMS and event-driven systems. Why listen to this podcast: - An event-driven system is a more modern approach to building highly scalable systems. - An RDBMS system can limit throughput in scalability. Camunda was able to achieve higher levels of scale by implementing an event-driven system. - Command and events are often confused. Commands are actions that request something to happen. Events describe something that happened. Confusing the two causes confusion in application development of event-driven systems.

In this podcast we discuss a holistic approach to technical leadership, and Pat provides guidance on everything from defining target operating models, cultivating culture, and supporting people in developing the career they would like. There are a bunch of great stories, several book recommendations, and additional resources to follow up on. * Cultivating organisational culture is much like gardening: you can’t force things, but you can set the right conditions for growth. The most effective strategy is to communicate the vision and goals, lead the people, and manage the systems and organisational structure. * N26, a challenger bank based in Berlin has experienced hypergrowth over the past two years. Both the number of customers and the amount of employees have increased over threefold. This provides lots of opportunities for ownership of product and projects, and it creates unique leadership challenges. * A target operating model (TOM) is a blueprint of a firm's business vision that aligns operating capacities and strategic objectives and provides an overview of the core business capabilities, internal factors, and external drivers, strategic and operational levers. This should be shared widely within an organisation * Pat has curated a “trident operating” model for employee growth. In addition to the class individual contributor (IC) and management tracks, he believes that a third “technical leadership” track provides many benefits. * People can switch between these tracks as their personal goals change. However, this switch can be challenging, and an organisation must support any transition with effective training. * Pat recommends the following books for engineers looking to make the transition to leadership: The Manager’s Path, by Camille Fournier; Resilient Management, by Lara Hogan; Elegant Puzzle, by Will Larson; and Leading Snowflakes by Oren Ellenbogen. Pat has also written his own book, Talking with Tech Leads. * It is valuable to define organisation values upfront. However, these can differ from actual culture, which more about what behaviours you allow, encourage, and stop. * Much like the values provided by Netflix’s Freedom and Responsibility model, Pat argues that balancing autonomy and alignment within an organisation is vital for success. Managers can help their team by clearly defining boundaries for autonomy and responsibility. * Developing the skills to influence people is very valuable for leaders. Influence is based on trust, and this must be constantly cultivated. Trust is much like a bank account, if you don’t regular deposit actions to build trust, you may find yourself going overdrawn when making a deposit. This can lead to bad will and defensive strategies being employed.

Cilium is open source software for transparently securing the network connectivity between application services deployed using Linux container management platforms like Docker and Kubernetes. It is a CNI plugin that offers layer 7 features typically seen with a service mesh. On this week’s podcast, Thomas Graf (one of the maintainers of Cilium and co-founder of Isovalent) discusses the recent 1.6 release, some of the security questions/concerns around eBPF, and the future roadmap for the project. Why listen to this podcast: * Cilium brings eBPF to the Cloud Native World. It works across both layer 4 and a layer 7. While it started as a pure eBPF plugin, they discovered that just caring about ports was not enough from a security perspective. * Cilium went 1.0 about a year and a half ago. 1.6 is the most featured-packed release of Cilium yet. Today, it has around 100 contributors. * While Cilium can make it much easier to manage IPTables, Cilium overlaps with a service mesh in that it can do things like understand application protocols, HTTP routes, or even restrict access to specific tables in data stores. * Cilium provides both in kernel and sidecar deployments. For sidecar deployments, it can work with Envoy to switch between kernel space and userspace code. The focus is on flexibility, performance, and low overhead. * BPF (Berkeley Packet Filter) was initial designed to do filtering on data links. eBPF has the same roots but it’s now used for system call filtering, tracing, sandbox, etc. It’s grown to be a general-purpose programming language to extend the Linux kernel. * Cilium has a multi-cluster feature built-in. The 1.6 release can run in a kube-proxy free configuration. It allows fine-grain network policies to run across multiple clusters without the use of IPTables. * Cilium offers on-the-wire encryption using in-kernel encryption technology that enables mTLS across all traffic in your service fleet. The encryption is completely transparent to the application. * eBPF has been used in all production environments at Facebook since May 2017. It’s been used at places like Netflix, Google, and Reddit. There are a lot of companies who have an interest in eBPF being secure and production-ready, so there’s a lot of attention focused on fixing and resolving and security issues that arise. * 1.6 also released KVstore-free operation, socket-based load balancing, CNI chaining, Native AWS ENI mode, enhancements to transparent encryption, and more. * The plans for 1.17 is to keep raising up the stack into the socket level (to offer things like load balancing and transparent encryption at scale) and likely offering deeper security features such as process-aware security policies for internal pod traffic. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2HCGnLa You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2HCGnLa

The three pillars of observability are logs, metrics, and tracing. Most teams are able to handle logs and metrics, while proper tracing can still be a challenge. On this podcast, we talk with Yuri Shkuro, the creator of Jaeger, author of the book Mastering Distributed Tracing, and a software engineer at Uber, about how the Jaeger tracing backend implements the OpenTracing API to handle distributed tracing. Why listen to the podcast: - Jaeger is an open-source tracing backend, developed at Uber. It also has a collection of libraries that implement the OpenTracing API. - At a high level, Jaeger is very similar to Zipkin, but Jaeger has features not available in Zipkin, including adaptive sampling and advanced visualization tools in the UI. - Tracing is less expensive than logging because data is sampled. It also gives you a complete view of the system. You can see a macro view of the transaction, and how it interacted with dozens of microservices, while still being able to drill down into the details of one service. - If you have only a handful of services, you can probably get away with logging and metrics, but once the complexity increases to dozens, hundreds, or thousands of microservices, you must have tracing. - Tracing does not work with a black box approach to the application. You can't simply use a service mesh then add a tracing framework. You need correlation between a single request and all the subsequent requests that it generates. A service mesh still relies on the underlying components handling that correlation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZlvMyR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZlvMyR

ROS is the Robotic Operating System. It’s been used by thousands of developers to prototype and create a robotic application. ROS can be found on robotics in warehouses, self-driving car companies, and on the International Space Station. Louise Poubel is an engineer working with Open Robotics. Today on the podcast, she talks about what it takes to develop software that moves in physical space, including the Sense, Think, Act Cycle, the developer experience, and architecture of ROS. Why listen to this podcast: - Writing code for robot development, you use the Sense, Think, Act Cycle. - ROS is an SDK for robotics. It provides a communication layer that enables data to flow between nodes that handle sensors, logic, and actuation. - ROS has two versions and has been around for twelve years. ROS 1 was entirely implemented in C. ROS 2 offers is a common C layer with implementations in many different languages, including Java, JavaScript, and Rust. - Released on a six-month cadence, Dashing was the latest release (May 2019). Previous releases were supported for one year, Dashing is the first LTS and will be supported for two years. - ROS 2 builds on top of the standard Data Distribution Service (DDS) that you find in mission-critical systems like nuclear power and airplanes. - Simulation is an important step in robotics. It allows you to prototype a system before deploying to a physical system. - Rviz is a three-dimensional visualizer used to visualize robots, the environments they work in, and sensor data. It is a highly configurable tool, with many different types of visualizations and plugins. It allows you to put together all your data in one place and see it. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZbOvrO You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZbOvrO

In this podcast we sit down with Matt Klein, software plumber at Lyft and creator of Envoy, and discuss topics including the continued evolution of the popular proxy, the strength of the open source Envoy community, and the value of creating and implementing standards throughout the technology stack. We also explore the larger topic of cloud natives platforms, and discuss the tradeoffs between using a simple and opinionated platform against something that is bespoke and more configurable, but also more complex. Related to this, Matt shares his thoughts on when and how to make the decision within an organisation to embrace technology like container orchestration and service meshes. Finally, we explore the creation of the new Envoy Mobile project. The goal of this project is to expand the capabilities provided by Envoy all the way out to mobile devices powered by Android and iOS. For example, most current user-focused traffic shifting that is conducted at the edge is implemented with coarse-grained approaches via by BGP and DNS, and using something like Envoy within mobile app networking stacks should allow finer-grained control. Why listen to this podcast: - The Envoy Proxy community has grown from strength-to-strength over the last year, from the inaugural EnvoyCon that ran alongside KubeCon NA 2018, to the increasing number of code contributions from engineers working across the industry - Attempting to create a community-driven “universal proxy data plane” with clearly defined APIs, like Envoy’s XDS API, has allowed vendors to collaborate on a shared abstraction while still allowing room for “differentiated success” to be built on top of this standard Google’s gRPC framework is adopting the Envoy XDS APIs, as this will allow both Envoy and gRPC instances to be operated via a single control plane, for example, Google Cloud Platform’s Traffic Director service. - There is a tendency within the software development industry to fetishise architectures that are designed and implemented by the unicorn tech companies, but not every organisation operates at this scale. - However, there has also been industry pushback against the complexity that modern platform components like container orchestration and service meshes can introduce to a technology stack. - Using a platform within these components provides the best return on investment when an organisation’s software architecture and development teams have reached a certain size. - Function-as-a-Service (Faas)-type platforms will most likely be how engineers will interact with software in the future. Business-focused developers often do not want to interact with the platform plumbing Envoy Mobile is building on prior art, and aims to expand the capabilities provided by Envoy all the way out to mobile devices using Android and iOS. Most current end user traffic shifting is implemented with coarse-grained approaches via BGP and DNS, and using something like Envoy instead will allow finer-grained control. - Using Envoy Mobile in combination with Protocol Buffers 3, which supports annotations on APIs, can facilitate working with APIs offline, configuring caching, and handling poor networking conditions. One of the motivations for this work is that small increases in application response times can lead to better business outcomes. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/33nlGMu You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/33nlGMu

On this podcast, we’re talking to Armon Dadgar, co-founder and CTO of HashiCorp. Alongside Mitchell Hashimoto, Armon founded HashiCorp over six years ago, and the company has gone from strength to strength, with their open source infrastructure product suite now consisting of Consul, Nomad, Vault and Terraform. We discuss the formation of the HashiCorp research division, and explore some of the computer science research underpinning Consul and Nomad. We also cover the challenges of supporting teams when they are looking to embrace new modes of working with dynamic infrastructure, and Armon introduces the new learn.hashicorp.com educational website and accompanying community and support forums. Why listen to this podcast: - There is a lot of fundamental computer science research that underpins the HashiCorp infrastructure workflow and configuration tooling. This helps to ensure that these mission-critical tools perform as expected, and enables sound reasoning about scaling these technologies. - The HashiCorp founders recognised the value of creating an industrial research-focused department within the company even when there were only 30 staff. - The Consul service mesh and distributed key value store leverages consensus and gossip algorithms from computer science research, Raft and SWIM, respectively. The HashiCorp team contributed a novel research-based improvement to SWIM -- Lifeguard: SWIM-ing with Situational Awareness -- that was presented at the DSN academic conference - Initially HashiCorp produced a new tool every 6-12 months, focusing on filling gaps within the infrastructure workflow tooling market. Now the focus is on refining the operator/user experience of the existing tools, creating more integrations with other platforms and tooling, and facilitating engineering teams adopting these tools, via the creation of educational resources and community forums. - Standardisation within computing technology can offer many benefits, especially where interoperability is required or technology switching costs are high. Care must be taken to ensure the correct interfaces are created, and that the time is right to create appropriate abstractions. - The HashiCorp team are focusing on "marching up the stack", with the goal that a lot of the underlying "plumbing" should be hidden from, or easily configurable by, application developers. This will allow developers to focus on adding value related to their business or organisation, rather than getting stuck with managing infrastructure. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2KptB3d You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2KptB3d

In this episode recorded at QCon London 2019 Shane Hastie, Lead Editor for Culture & Methods, first spoke to Kingsley Davies about ethics and then with Cat Swetel about requisite variety and being mindful of the impact our decisions have for the future. Why listen to this podcast: • The need to explore the application of technology for good • The need for ethical standards in the technology industry • Data is the new oil and it is frequently used in ways that are not in the best interest of society • Other engineering professions have codes of conduct and ethical frameworks that are mandated as part of the education process, software engineering currently has very little • Ashby’s law of requisite variety – the more options that are available to a system, the more resilient the system is applies to all aspects of our socio-technical systems • We exist in the realm of ethics – we can’t just go to work and do what you’re told. Everything we do is a choice and our choices have a huge impact on the future

The promise of Java has always been, “write once, run anywhere.” This was enabled through just-in-time compilation, which allowed developers to target a platform at compilation. But, this flexibility has given rise to comments like, “Java is slow.” What if you could compile Java to Native Code? On this podcast, we’re talking to Thomas Wuerthinger, a senior research director at Oracle Labs. Leading programming language implementation teams for Java, JavaScript, Ruby, and R. He is the architect of the Graal compiler and the Truffle self-optimizing runtime. Why listen to the podcast: - The GraalVM project was initially just a replacement for the JVM C2 just-in-time compiler, but has evolved to include support for multiple languages, as well as an ahead-of-time compiling mode. - Support for multiple languages can provide better performance for some languages, as well as making direct calls without inter-process communication. - With GraalVM’s AOT compilation, you can statically link system libraries, which allows you to run a static binary on a bare-metal Docker image, without even a Linux distribution. - The major benefits of AOT are minimized startup time, memory footprint, and packaging size. This can come with a trade-off in reduced maximum throughput and higher latency. - The GraalVM roadmap includes supporting additional platforms, such as Windows and mobile, as well as performance improvements for both the JIT and AOT compilers. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Y2hPk2 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Y2hPk2

On this podcast, Wes talks to John Xmas. Johnny works for Kasada, a company that offers a security platform to help ensure only your users are logging into your web applications. Johnny is a well-known figure in the security space. The two discuss common attack vectors, the OWASP Top 10, and then walk through what hackers commonly do attempting to compromise a system. The show is full of advice on protecting your systems including topics around Defense in Depth, Time-Based Security, two-factor authentication, logging/alerting, security layers, and much more. Why listen to this podcast: - While there are sophisticated web attacks out there that use things like PhantomJS or Headless Chome, the vast majority of the web application attacks are the same unsophisticated scripted attacks that you always hear about. These are simple scripts using tools like curl and BurpSuite with Python or JavaScript. These simple scripts are still incredibly effective. - OWASP Top 10 really hasn’t changed all that much in the last ten years. For example, despite being the number one approach used to educate defensive engineers on how to protect their apps, SQLI (SQL Injection) is still the most common attack. We continue to repeat the same mistakes that have exposed systems for a decade now. - Phishing is by and far the quickest way to compromise a system. Defensive in Depth, security boundaries, limiting local admin rights are all things that corporations can implement to minimize the blast radius. - Attackers have hundreds of gigs of actual username/password combinations that have been exposed from all the breaches over the past few years. These are often a first step when attempting to compromise a system. It’s more often likely that they will figure out a valid email pattern for a company and then feed actual names into that pattern to go after the username. From there, brute force attacks with those usernames against libraries of passwords is a common approach. - A common approach is to go after an email login. While the email can be a treasure trove of information, it’s more about using those credentials in other places. It’s pretty common, for example, to use those credentials to get into a network with a VPN. - Captcha/reCaptcha is not very effective and preventing these brute force attacks. There are a large number of bypasses and even Mechanical Turk companies that are available to bypass these tools. What can be effective is Time Based Security because it slows the attackers down. If you can slow them down, you can make the attack say long to succeed that they’ll go somewhere else. - Once inside the network, most companies often have little security on internal systems. Multi-factor authentication, not just on the front door, but on internal systems is a huge step in the right direction. Monitoring not only for failed login attempts but, in some situations, valid login attempts (such as when a domain admin logs into a domain controller) should absolutely be used. - When it comes to application security between services within a network, the best advice is to make sure developers really understand what is trying to be accomplished by something like JWT (JSON Web Tokens). Often its the lack of understanding of what they’re actually doing that leads to system vulnerabilities. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2MSIAXG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2MSIAXG

Today on the podcast, Wes talks with Mike Milinkovich, Executive Director of the Eclipse Foundation. The Eclipse Foundation was chosen to govern the evolution of Oracle’s Java EE to Jakarta EE. The two discuss the project, the recent news about issues with the javax namespace, the challenges around bundling a Java Runtime with Eclipse, and the path forward for Jakarta EE 9 and beyond. Why listen to this podcast: - Java EE, unlikely Java SE, has always been a multi-vendor ecosystem. It made sense for everyone for Oracle to invite their partners to be involved in the governance of the specification for Java EE for it to continue moving forward. This is the reason for moving Java EE into the Eclipse Foundation as Jakarta EE. - The current plan is for the Eclipse Foundation to get a copyright license to evolve the text of the specification and not a license to the trademarks of Java EE itself. - The javax namespace must remain as is. For it to be evolved, a different namespace must be used. The javax namespace is a trademark of Oracle. Because of this, there are quality controls that Oracle required for its evolution. Ultimately because of those controls, the Eclipse Foundation felt it was better to branch javax into a different namespace and evolve it separately solely under Jakarta EE governance. - Jakarta EE 8 is targeted to be released around Oracle Code ONE. Jakarta EE 8 will be exactly the same as Java EE 8. * The only difference is it will be licensed from Jakarta, not Oracle and only requires membership in the Working Group. - Beyond EE 8, the release cycle, the plan for moving the javax namespace (and keeping compatibility with both the old javax namespace and the new namespace), and new specifications for inclusion into Jakarta EE are still active areas of discussion. - Unrelated to the discussion of Jakarta EE (but discussed in the same board meeting), an attempt to bundle OpenJ9 with the Eclipse IDE failed because of licensing restrictions around a certified Java Runtime. OpenJ9 is certified when acquired through an IBM channel, but not when downloaded directly for us. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2HSfcfM You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2HSfcfM