The New Stack Podcast

LLM observability focuses on maximizing the utility of larger language models (LLMs) by monitoring key metrics and signals. Alex Williams, Founder and Publisher for The New Stack, and Janikiram MSV, Principal of Janikiram & Associates and an analyst and writer for The New Stack, discusses the emergence of the LLM stack, which encompasses various components like LLMs, vector databases, embedding models, retrieval systems, read anchor models, and more. The objective of LLM observability is to ensure that users can extract desired outcomes effectively from this complex ecosystem.Similar to infrastructure observability in DevOps and SRE practices, LLM observability aims to provide insights into the LLM stack's performance. This includes monitoring metrics specific to LLMs, such as GPU/CPU usage, storage, model serving, change agents in applications, hallucinations, span traces, relevance, retrieval models, latency, monitoring, and user feedback. MSV emphasizes the importance of monitoring resource usage, model catalog synchronization with external providers like Hugging Face, vector database availability, and the inference engine's functionality.He also mentions peer companies in the LLM observability space like Datadog, New Relic, Signoz, Dynatrace, LangChain (LangSmith), Arize.ai (Phoenix), and Truera, hinting at a deeper exploration in a future episode of The New Stack Makers. Learn more from The New Stack about LLM and observability Observability in 2024: More OpenTelemetry, Less Confusion How AI Can Supercharge Observability Next-Gen Observability: Monitoring and Analytics in Platform Engineering Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

In a conversation on The New Stack Makers, co-hosted by Alex Williams, TNS founder and publisher, and Charles Humble, an industry expert who served as a software engineer, architect and CTO and now podcaster, author and consultant at Conissaunce Ltd., discussed why software developers and engineers should care about their impact on climate change. Humble emphasized that building software sustainably starts with better operations, leading to cost savings and improved security. He cited past successes in combating environmental issues like acid rain and the ozone hole through international agreements and emissions reduction strategies.Despite modest growth since 2010, data centers remain significant electricity consumers, comparable to countries like Brazil. The power-intensive nature of AI models exacerbates these challenges and may lead to scarcity issues. Humble mentioned the Green Software Foundation's Maturity Matrix with goals for carbon-free data centers and longer device lifespans, discussing their validity and the role of regulation in achieving them. Overall, software development's environmental impact, primarily carbon emissions, necessitates proactive measures and industry-wide collaboration. Learn more from The New Stack about sustainability: What is GreenOps? Putting a Sustainable Focus on FinOpsUnraveling the Costs of Bad Code in Software Development Can Reducing Cloud Waste Help Save the Planet?How to Build Open Source Sustainability Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

This New Stack Makers podcast co-hosted by Alex Williams, TNS founder and publisher, and Adrian Cockcroft, Partner and Analyst at OrionX.net, discussed Nvidia's GH200 Grace Hopper superchip. Industry expert Sunil Mallya, Co-founder and CTO of Flip AI weighed in on how it is revolutionizing the hardware industry for AI workloads by centralizing GPU communication, reducing networking overhead, and creating a more efficient system. Mallya noted that despite its innovative design, challenges remain in adoption due to interface issues and the need for software to catch up with hardware advancements. However, optimism persists for the future of AI-focused chips, with Nvidia leading the charge in creating large-scale coherent memory systems. Meanwhile, Flip AI, a DevOps large language model, aims to interpret observability data to troubleshoot incidents effectively across various cloud platforms. While discussing the latest chip innovations and challenges in training large language models, the episode sheds light on the evolving landscape of AI hardware and software integration.Learn more from The New Stack about Nvidia and the future of chip design Nvidia Wants to Rewrite the Software Development Stack Nvidia GPU Dominance at a Crossroads Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

This New Stack Makers podcast co-hosted by TNS founder and publisher, Alex Williams and Joan Westenberg, founder and writer of Joan’s Index, discussed Copilot. Westenberg highlighted its integration with Microsoft 365 and its role as a coding assistant, showcasing its potential to streamline various tasks. However, she also revealed its limitations, particularly in reliability. Despite being designed to assist with tasks across Microsoft 365, Copilot's performance fell short during Westenberg's tests, failing to retrieve necessary information from her email and Microsoft Teams meetings. While Copilot proves useful for coding, providing helpful code snippets, its effectiveness diminishes for more complex projects. Westenberg's demonstrations underscored both the strengths and weaknesses of Copilot, emphasizing the need for improvement, especially in reliability, to fulfill its promise as a versatile work companion. Learn more from The New Stack about Copilot Microsoft One-ups Google with Copilot Stack for Developers Copilot Enterprises Introduces Search and Customized Best Practices Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

This New Stack Makers podcast co-hosted by Adrian Cockroft, analyst at OrionX.net and TNS founder and publisher, Alex Williams discusses the importance of monitoring services utilizing Large Language Models (LLMs) and the emergence of tools like LangChain and LangSmith to address this need. Adrian Cockcroft, formerly of Netflix and now working with The New Stack, highlights the significance of monitoring AI apps using LLMs and the challenges posed by slow and expensive API calls from LLMs. LangChain acts as middleware, connecting LLMs with services, akin to the Java Database Controller. LangChain's monitoring capabilities led to the development of LangSmith, a monitoring tool. Another tool, LangKit by WhyLabs, offers similar functionalities but is less integrated. This reflects the typical evolution of open-source projects into commercial products. LangChain recently secured funding, indicating growing interest in such monitoring solutions. Cockcroft emphasizes the importance of enterprise-level support and tooling for integrating these solutions into commercial environments. This discussion underscores the evolving landscape of monitoring services powered by LLMs and the emergence of specialized tools to address associated challenges. Learn more from The New Stack about LangChain: LangChain: The Trendiest Web Framework of 2023, Thanks to AI How Retool AI Differs from LangChain (Hint: It's Automation) Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

In this New Stack Makers podcast, Martin Parker, a solutions architect for UST, spoke with TNS editor-in-chief, Heather Joslyn and discussed the significance of internal developer platforms (IDPs), emphasizing benefits beyond frontend developers to backend engineers and site reliability engineers (SREs). Parker highlighted the role of IDPs in automating repetitive tasks, allowing SREs to focus on optimizing application performance. Standardization is key, ensuring observability and monitoring solutions align with best practices and cater to SRE needs. By providing standardized service level indicators (SLIs) and key performance indicators (KPIs), IDPs enable SREs to maintain reliability efficiently. Parker stresses the importance of avoiding siloed solutions by establishing standardized practices and tools for effective monitoring and incident response. Overall, the deployment of IDPs aims to streamline operations, reduce incidents, and enhance organizational value by empowering SREs to concentrate on system maintenance and improvements.Learn more from The New Stack about UST: Cloud Cost-Unit Economics- A Modern Profitability Model Cloud Native Users Struggle to Achieve Benefits, Report Says John our community of newsletter subscribers to stay on top of the news and at the top of your game.

In this New Stack Makers podcast, Ben Wilcock, a senior technical marketing architect for Tanzu, spoke with TNS editor-in-chief, Heather Joslyn and discussed the challenges organizations face when building internal developer platforms, particularly the issue of scope, at KubeCon + CloudNativeCon North America. He emphasized the difficulty for platform engineering teams to select and integrate various Kubernetes projects amid a plethora of options. Wilcock highlights the complexity of tracking software updates, new features, and dependencies once choices are made. He underscores the advantage of having a standardized approach to software deployment, preventing errors caused by diverse mechanisms. Tanzu aims to simplify the adoption of platform engineering and internal developer platforms, offering a turnkey approach with the Tanzu Application Platform. This platform is designed to be flexible, malleable, and functional out of the box. Additionally, Tanzu has introduced the Tanzu Developer Portal, providing a focal point for developers to share information and facilitating faster progress in platform engineering without the need to integrate numerous open source projects. Learn more from The New Stack about Tanzu and internal developer platforms:VMware Unveils a Pile of New Data Services for Its Cloud VMware VMware Expands Tanzu into a Full Platform Engineering Environment VMware Targets the Platform Engineer Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

In this New Stack Makers podcast, Mike Stefaniak, senior product manager at NGINX and Kate Osborn, a software engineer at NGINX discusses challenges associated with network ingress in Kubernetes clusters and introduces the Kubernetes Gateway API as a solution. Stefaniak highlights the issues that arise when multiple teams work on the same ingress, leading to friction and incidents. NGINX has also introduced the NGINX Gateway Fabric, implementing the Kubernetes Gateway API as an alternative to network ingress. The Kubernetes Gateway API, proposed four years ago and recently made generally available, offers advantages such as extensibility. It allows referencing policies with custom resource definitions for better validation, avoiding the need for annotations. Each resource has an associated role, enabling clean application of role-based access control policies for enhanced security.While network ingress is prevalent and mature, the Kubernetes Gateway API is expected to find adoption in greenfield projects initially. It has the potential to unite North-South and East-West traffic, offering a role-oriented API for comprehensive control over cluster traffic. The article encourages exploring the Kubernetes Gateway API and engaging with the community to contribute to its development.Learn more from The New Stack about NGINX and the open source Kubernetes Gateway API:Kubernetes API Gateway 1.0 Goes Live, as Maintainers Plan for The Future API Gateway, Ingress Controller or Service Mesh: When to Use What and Why Ingress Controllers or the Kubernetes Gateway API? Which is Right for You? Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

TNS publisher Alex Williams spoke with Ben Kramer, co-founder and CTO of Monterey.ai Cole Hoffer, Senior Software Engineer at Monterey.ai to discuss how the company utilizes vector search to analyze user voices, feedback, reviews, bug reports, and support tickets from various channels to provide product development recommendations. Monterey.ai connects customer feedback to the development process, bridging customer support and leadership to align with user needs. Figma and Comcast are among the companies using this approach. In this interview, Kramer discussed the challenges of building Large Language Model (LLM) based products and the importance of diverse skills in AI web companies and how Monterey employs Zilliz for vector search, leveraging Milvus, an open-source vector database. Kramer highlighted Zilliz's flexibility, underlying Milvus technology, and choice of algorithms for semantic search. The decision to choose Zilliz was influenced by its performance in the company's use case, privacy and security features, and ease of integration into their private network. The cloud-managed solution and Zilliz's ability to meet their needs were crucial factors for Monterey AI, given its small team and preference to avoid managing infrastructure.Learn more from The New Stack about Zilliz and vector database search:Improving ChatGPT’s Ability to Understand Ambiguous PromptsCreate a Movie Recommendation Engine with Milvus and PythonUsing a Vector Database to Search White House Speeches Join our community of newsletter subscribers to stay on top of the news and at the top of your game. https://thenewstack.io/newsletter/

TNS host Heather Joslyn sits down with Ron Masas to discuss trade-offs when it comes to creating fast, secure applications and APIs. He notes a common issue of neglecting documentation and validation, leading to vulnerabilities. Weak authorization is a recurring problem, with instances where changing an invoice ID could expose another user's data.Masas, an ethical hacker, highlights the risk posed by "zombie" APIs—applications that have become disused but remain potential targets. He suggests investigating frameworks, checking default configurations, and maintaining robust logging to enhance security. Collaboration between developers and security teams is crucial, with "security champions" in development teams and nuanced communication about vulnerabilities from security teams being essential elements for robust cybersecurity.For further details, the podcast discusses case studies involving TikTok and Digital Ocean, Masas's views on AI and development, and anticipated security challenges.Learn more from The New Stack about Imperva and API security:What Developers Need to Know about Business Logic AttacksWhy Your APIs Aren’t Safe — and What to Do about ItThe Limits of Shift-Left: What’s Next for Developer Security Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

Platform engineering “is the art of designing and binding all of the different tech and tools that you have inside of an organization into a golden path that enables self service for developers and reduces cognitive load,” said Kaspar Von Grünberg, founder and CEO of Humanitec, in this episode of The New Stack Makers podcast.This structure is important for individual contributors, Grünberg said, as well as backend engineers: “if you look at the operation teams, it reduces their burden to do repetitive things. And so platform engineers build and design internal developer platforms, and help and serve users."This conversation, hosted by Heather Joslyn, TNS features editor, dove into platform engineering: what it is, how it works, the problems it is intended to solve, and how to get started in building a platform engineering operation in your organization. It also debunks some key fallacies around the concept.Learn more from The New Stack about Platform Engineering and Humanitec:Platform Engineering Overview, News, and TrendsThe Hype Train Is Over. Platform Engineering Is Here to Stay9 Steps to Platform Engineering Hell

Is the end of programming nigh? That's the big question posed in this episode recorded earlier in 2023. It was very popular among listeners, and with the topic being as relevant as ever, we wanted to wrap up the year by highlighting this conversation again.If you ask Matt Welsh, he'd say yes, the end of programming is upon us. As Richard McManus wrote on The New Stack, Welsh is a former professor of computer science at Harvard who spoke at a virtual meetup of the Chicago Association for Computing Machinery (ACM), explaining his thesis that ChatGPT and GitHub Copilot represent the beginning of the end of programming.Welsh joined us on The New Stack Makers to discuss his perspectives about the end of programming and answer questions about the future of computer science, distributed computing, and more.Welsh is now the founder of fixie.ai, a platform they are building to let companies develop applications on top of large language models to extend with different capabilities.For 40 to 50 years, programming language design has had one goal. Make it easier to write programs, Welsh said in the interview.Still, programming languages are complex, Welsh said. And no amount of work is going to make it simple. Learn more from The New Stack about AI and the future of software development:Top 5 Large Language Models and How to Use Them Effectively30 Non-Trivial Ways for Developers to Use GPT-4Developer Tips in AI Prompt Engineering

Kubevirt, a relatively new capability within Kubernetes, signifies a shift in the virtualization landscape, allowing operations teams to run KVM virtual machines nested in containers behind the Kubernetes API. This integration means that the Kubernetes API now encompasses the concept of virtual machines, enabling VM-based workloads to operate seamlessly within a cluster behind the API. This development addresses the challenge of transitioning traditional virtualized environments into cloud-native settings, where certain applications may resist containerization or require substantial investments for adaptation.The emerging era of virtualization simplifies the execution of virtual machines without concerning the underlying infrastructure, presenting various opportunities and use cases. Noteworthy advantages include simplified migration of legacy applications without the need for containerization, thereby reducing associated costs.Kubevirt 1.1, discussed at KubeCon in Chicago by Red Hat's Vladik Romanovsky and Nvidia's Ryan Hallisey, introduces features like memory hotplug and vCPU hotplug, emphasizing the stability of Kubevirt. The platform's stability now allows for the implementation of features that were previously constrained.Learn more from The New Stack about Kubevirt and the Cloud Native Computing Foundation:The Future of VMs on Kubernetes: Building on KubeVirtA Platform for KubernetesScaling Open Source Community by Getting Closer to Users

The Kubernetes landscape is evolving, shifting from the domain of visionaries and early adopters to a more mainstream audience. Tigera, represented by CEO Ratan Tipirneni at KubeCon North America in Chicago, recognizes the changing dynamics and the demand for simplified Kubernetes solutions. Tigera's open-source Calico security platform has been updated with a focus on mainstream users, presenting a cohesive and user-friendly solution. This update encompasses five key capabilities: vulnerability scoring, configuration hardening, runtime security, network security, and observability.The aim is to provide users with a comprehensive view of their cluster's security through a zero to 100 scoring system, tracked over time. Tigera's recommendation engine suggests actions to enhance overall security based on the risk profile, evaluating factors such as egress traffic controls and workload isolation within dynamic Kubernetes environments. Tigera emphasizes the importance of understanding the actual flow of data across the network, using empirical data and observed behavior to build accurate security measures rather than relying on projections. This approach addresses the evolving needs of customers who seek not just vulnerability scores but insights into runtime behavior for a more robust security profile.Learn more from The New Stack about Tigera and Cloud Native Security:Cloud Native Network Security: Who’s Responsible?Turbocharging Host Workloads with Calico eBPF and XDP3 Observability Best Practices for Cloud Native App Security

Boeing, with around 6,000 engineers, is emphasizing open source engagement by focusing on three main themes, according to Damani Corbin, who heads Boeing's Open Source office. He joined our host, Alex Williams, for a discussion at KubeCon+CloudNativeCon in Chicago.The first priority Corbin talks about is simplifying the consumption of open source software for developers. Second, Boeing aims to facilitate developer contributions to open source projects, fostering involvement in communities like the Cloud Native Computing Foundation and the Linux Foundation. The third theme involves identifying opportunities for "inner sourcing" to share internally developed solutions across different groups.Boeing is actively working to break down barriers and encourage code reuse across the organization, promoting participation in open source initiatives. Corbin highlights the importance of separating business-critical components from those that can be shared with the community, prioritizing security and extending efforts to enhance open source security practices. The organization is consolidating its open source strategy by collaborating with legal and information security teams.Corbin emphasizes the goal of making open source involvement accessible and attractive, with a phased approach to encourage meaningful contributions and ultimately enabling the compensation of engineers for open source work in the future.Learn more from The New Stack about Boeing and CNCF open source projects:How Boeing Uses Cloud NativeHow Open Source Has Turned the Tables on Enterprise SoftwareScaling Open Source Community by Getting Closer to UsersMercedes-Benz: 4 Reasons to Sponsor Open Source Projects

At KubeCon + CloudNativeCon North America 2022, Amazon Web Services (AWS) revealed plans to mirror Kubernetes assets hosted on Google Cloud, addressing Cloud Native Computing Foundation's (CNCF) egress costs. A year later, the project, led by AWS's Davanum Srinivas, redirects image requests to the nearest cloud provider, reducing egress costs for users.AWS's Todd Neal and Jonathan Innis discussed this on The New Stack Makers podcast recorded at KubeCon North America 2023. Neal explained the registry's functionality, allowing users to pull images directly from the respective cloud provider, avoiding egress costs.The discussion also highlighted AWS's recent open source contributions, including beta features in Kubectl, prerelease of Containerd 2.0, and Microsoft's support for Karpenter on Azure. Karpenter, an AWS-developed Kubernetes cluster autoscaler, simplifies node group configuration, dynamically selecting instance types and availability zones based on running pods.The AWS team encouraged developers to contribute to Kubernetes ecosystem projects and join the sig-node CI subproject to enhance kubelet reliability. The conversation in this episode emphasized the benefits of open development for rapid feedback and community collaboration.Learn more from The New Stack about AWS and Open Source:Powertools for AWS Lambda Grows with Help of VolunteersAmazon Web Services Open Sources a KVM-Based Fuzzing FrameworkAWS: Why We Support Sustainable Open Source

In the past year, developers have faced both promise and uncertainty, particularly in the realm of generative AI. Heath Newburn, global field CTO for PagerDuty, joins TNS host Heather Joslyn to talk about the impact AI and other topics will have on developers in 2024.Newburn anticipates a growing emphasis on DevSecOps in response to high-profile cyber incidents, noting a shift in executive attitudes toward security spending. The rise of automation-centric tools like Backstage signals a changing landscape in the link between development and operations tools. Notably, there's a move from focusing on efficiency gains to achieving new outcomes, with organizations seeking innovative products rather than marginal coding speed improvements.Newburn highlights the importance of experimentation, encouraging organizations to identify areas for trial and error, learning swiftly from failures. The upcoming year is predicted to favor organizations capable of rapid experimentation and information gathering over perfection in code writing.Listen to the full podcast episode as Newburn further discusses his predictions related to platform engineering, remote work, and the continued impact of generative AI.Learn more from The New Stack about PagerDuty and trends in software development:How AI and Automation Can Improve Operational ResiliencyWhy Infrastructure as Code Is Vital for Modern DevOpsOperationalizing AI: Accelerating Automation, DataOps, AIOps

In this episode of The New Stack Makers, Rob Skillington, co-founder and CTO of Chronosphere, discusses the challenges engineers face in building tools for their organizations. Skillington emphasizes that the "build or buy" decision oversimplifies the issue of tooling and suggests that understanding the abstractions of a project is crucial. Engineers should consider where to build and where to buy, creating solutions that address the entire problem. Skillington advises against short-term thinking, urging innovators to consider the long-term landscape.Drawing from his experience at Uber, Skillington highlights the importance of knowing the audience and customer base, even when they are colleagues. He shares a lesson learned when building a visualization platform for engineers at Uber, where understanding user adoption as a key performance indicator upfront could have improved the project's outcome.Skillington also addresses the "not invented here syndrome," noting its prevalence in organizations like Microsoft and its potential impact on tool adoption. He suggests that younger companies, like Uber, may be more inclined to explore external solutions rather than building everything in-house. The conversation provides insights into Skillington's experiences and the considerations involved in developing internal tools and platforms.Learn more from The New Stack about Software Engineering, Observability, and Chronosphere:Cloud Native Observability: Fighting Rising Costs, IncidentsA Guide to Measuring Developer Productivity 4 Key Observability Best Practices

Jean Yang, founder of API observability company Akita Software, emphasizes that programming languages should be shaped by software development needs and data, rather than philosophical ideals. Yang, a former assistant professor at Carnegie Mellon University, believes that programming tools and processes should be influenced by actual use and data, prioritizing the developer experience over the language creator's beliefs. With a background in programming languages, Yang advocates for a shift away from the outdated notion that language developers are building solely for themselves.In this discussion on The New Stack Makers, Yang underscores the importance of understanding the reality of developers' needs, especially as developer tools have evolved into a full-time industry. She argues for a focus on UX design and product fundamentals in developing tools, moving beyond the traditional mindset where developer tools were considered side projects.Yang founded Akita to address the challenges of building reliable software systems in a world dominated by APIs and microservices. The company transitioned to API observability, recognizing the crucial role APIs play in enhancing the understandability of complex systems. Yang's commitment to improving software correctness and the belief in APIs as key to abstraction and ease of monitoring align with Postman's direction after acquiring Akita. Postman aims to serve developers worldwide, emphasizing the significance of APIs in complex systems.Check out more episodes from The Tech Founder Odyssey series:How Byteboard’s CEO Decided to Fix the Broken Tech InterviewA Lifelong ‘Maker’ Tackles a Developer Onboarding ProblemHow Teleport’s Leader Transitioned from Engineer to CEO

Docker CTO Justin Cormack reveals that Docker has been a go-to tool for data scientists in AI and machine learning for years, primarily in specialized areas like image processing and prediction models. However, the release of OpenAI's ChatGPT last year sparked a significant surge in Docker's popularity within the AI community.The focus shifted to large language models (LLMs), with a growing interest in the retrieval-augmented generation (RAG) stack. Docker's collaboration with Ollama enables developers to run Llama 2 and Code Llama locally, simplifying the process of starting and experimenting with AI applications. Additionally, partnerships with Neo4j and LangChain allow for enhanced support in storing and retrieving data for LLMs.Cormack emphasizes the simplicity of getting started locally, addressing challenges related to GPU shortages in the cloud. Docker's efforts also include building an AI solution using its data, aiming to assist users in Dockerizing applications through an interactive notebook in Visual Studio Code. This tool leverages LLMs to analyze applications, suggest improvements, and generate Docker files tailored to specific languages and applications.Docker's integration with AI technologies demonstrates a commitment to making AI and Docker more accessible and user-friendly.Learn more from The New Stack about AI and Docker:Artificial Intelligence News, Analysis, and ResourcesWill GenAI Take Jobs? No, Says Docker CEODebugging Containers in Kubernetes — It’s Complicated

In this episode, Stefano Maffulli, Executive Director of the Open Source Initiative, discusses the need for a new definition as AI differs significantly from open source software. The complexity arises from the unique nature of AI, particularly large language models and transformers, which challenge traditional copyright frameworks. Maffulli emphasizes the urgency of establishing a definition for open source AI and discusses an ongoing effort to release a set of principles by the year's end.The concept of "open" in the context of AI is undergoing a significant transformation, reminiscent of the early days of open source. The recent upheaval at OpenAI, resulting in the removal of CEO Sam Altman, reflects a profound shift in the technology community, prompting a reconsideration of the definition of "open" in the realm of AI.The conversation highlights the parallels between the current AI debate and the early days of software development, emphasizing the necessity for a cohesive approach to navigate the evolving landscape. Altman's ousting underscores a clash of belief systems within OpenAI, with a "safetyist" community advocating caution and transparency, while Altman leans towards experimentation. The historical significance of open source, with a focus on trust preservation over technical superiority, serves as a guide for defining "open" and "AI" in a rapidly changing environment.Learn more from The New Stack about AI and Open Source:Artificial Intelligence News, Analysis, and ResourcesOpen Source Development Threatened in EuropeThe AI Engineer Foundation: Open Source for the Future of AI

DockerCon showcased a commitment to enhancing the developer experience, with a particular focus on addressing the challenge of debugging containers in Kubernetes. The newly launched Docker Debug offers a language-independent toolbox for debugging both local and remote containerized applications.By abstracting Kubernetes concepts like pods and namespaces, Docker aims to simplify debugging processes and shift the focus from container layers to the application itself. Our guest, Docker Principal Engineer Ivan Pedrazas, emphasized the need to eliminate unnecessary complexities in debugging, especially in the context of Kubernetes, where developers grapple with unfamiliar concerns exposed by the API.Another Docker project, Tape, simplifies deployment by consolidating Kubernetes artifacts into a single package, streamlining the process for developers. The ultimate goal is to facilitate debugging of slim containers with minimal dependencies, optimizing security and user experience in Kubernetes development.While progress is being made, bridging the gap between developer practices and platform engineering expectations remains an ongoing challenge.Learn more from The New Stack about Kubernetes and Docker:Kubernetes Overview, News, and TrendsDocker Rolls out 3 Tools to Speed and Ease DevelopmentWill GenAI Take Jobs? No, Says Docker CEO

TNS host Alex Williams is joined by Florian Valeye, a data engineer at Back Market, to shed light on the evolving landscape of data engineering, particularly focusing on Delta Lake and his contributions to open source communities. As a member of the Delta Lake community, Valeye discusses the intersection of data warehouses and data lakes, emphasizing the need for a unified platform that breaks down traditional barriers.Delta Lake, initially created by Databricks and now under the Linux Foundation, aims to enhance reliability, performance, and quality in data lakes. Valeye explains how Delta Lake addresses the challenges posed by the separation of data warehouses and data lakes, emphasizing the importance of providing asset transactions, real-time processing, and scalable metadata.Valeye's involvement in Delta Lake began as a response to the challenges faced at Back Market, a global marketplace for refurbished devices. The platform manages large datasets, and Delta Lake proved to be a pivotal solution in optimizing ETL processes and facilitating communication between data scientists and data engineers.The conversation delves into Valeye's journey with Delta Lake, his introduction to Rust programming language, and his role as a maintainer in the Rust-based library for Delta Lake. Valeye emphasizes Rust's importance in providing a high-level API with reliability and efficiency, offering a balanced approach for developers.Looking ahead, Valeye envisions Delta Lake evolving beyond traditional data engineering, becoming a platform that seamlessly connects data scientists and engineers. He anticipates improvements in data storage optimization and envisions Delta Lake serving as a standard format for machine learning and AI applications.The conversation concludes with Valeye reflecting on his future contributions, expressing a passion for Rust programming and an eagerness to explore evolving projects in the open-source community. Learn more from The New Stack about Delta Lake and The Linux Foundation:Delta Lake: A Layer to Ensure Data QualityData in 2023: Revenge of the SQL NerdsWhat Do You Know about Your Linux System?

Liam Crilly, Senior Director of Product Management at NGINX, discussed the potential of WebAssembly (Wasm) during this recording at the Open Source Summit in Bilbao, Spain. With over three decades of experience, Crilly highlighted WebAssembly's promise of universal portability, allowing developers to build once and run anywhere across a network of devices.While Wasm is more mature on the client side in browsers, its deployment on the server side is less developed, lacking sufficient runtimes and toolchains. Crilly noted that WebAssembly acts as a powerful compiler target, enabling the generation of well-optimized instruction set code. Despite the need for a virtual machine, WebAssembly's abstraction layer eliminates hardware-specific concerns, providing near-native compute performance through additional layers of optimization.Learn more from The New Stack about WebAssembly and NGINX:WebAssembly Overview, News and TrendsWhy WebAssembly Will Disrupt the Operating SystemTrue Portability Is the Killer Use Case for WebAssembly4 Factors of a WebAssembly Native World

Jonathan Katz, a principal product manager at Amazon Web Services, discusses the evolution of PostgreSQL in an episode of The New Stack Makers. He notes that PostgreSQL's uses have expanded significantly since its inception and now cover a wide range of applications and workloads. Initially considered niche, it faced competition from both open-source and commercial relational database systems. Katz's involvement in the PostgreSQL community began as an app developer, and he later contributed by organizing events.PostgreSQL originated from academic research at the University of California at Berkeley in the mid-1980s, becoming an open-source project in 1994. In the mid-1990s, proprietary databases like Oracle, IBM DB2, and Microsoft SQL dominated the market, while open-source alternatives like MySQL, MariaDB, and SQLite emerged.PostgreSQL 16 introduces logical replication from standby servers, enhancing scalability by offloading work from the primary server. The meticulous design process within the PostgreSQL community leads to stable and reliable features. Katz mentions the development of Direct I/O as a long-term feature to reduce latency and improve data writing performance, although it will take several years to implement.Amazon Web Services has built Amazon RDS on PostgreSQL to simplify application development for developers. This managed service handles operational tasks such as deployment, backups, and monitoring, allowing developers to focus on their applications. Amazon RDS supports multiple PostgreSQL releases, making it easier for businesses to manage and maintain their databases.Learn more from The New Stack about PostgreSQL and AWS:PostgreSQL 16 Expands Analytics CapabilitiesPowertools for AWS Lambda Grows with Help of VolunteersHow Donating Open Source Code Can Advance Your Career

The practice of "shift left," which involves moving security concerns to the code level and increasing developers' responsibility for security, is facing a backlash, with both developers and security professionals expressing concerns. Peter Klimek, director of technology at Imperva, discusses the reasons behind this backlash in this episode.Some organizations may have exhausted the benefits of shift left, while the main challenge for many isn't finding vulnerabilities but finding time to address them. Security attacks are now targeting business logic vulnerabilities rather than dependencies, which shift left tools are better at identifying. These business logic vulnerabilities are often tied to authorization decisions, making them harder to address through code-level tools. Additionally, attacks increasingly focus on the frontend, such as API development and cart attacks.Klimek emphasizes the need for development and security teams to collaborate and advocates for using DORA metrics to assess the impact of security efforts on the development pipeline. Some organizations may reach a point where the tools added to the development lifecycle become counterproductive, he notes. DORA metrics can help determine when this occurs and provide valuable insights for security teams.Learn more from The New Stack about Developer Security and Imperva:Why Your APIs Aren’t Safe — and What to Do about ItWhat Developers Need to Know about Business Logic AttacksAre Your Development Practices Introducing API Security Risks?

Operational resiliency, as explained by Dormain Drewitz of PagerDuty, involves the ability to bounce back and recover from setbacks, not only technically but also in terms of organizational recovery. True resiliency means maintaining the willingness to take risks even after facing challenges. In a conversation with Heather Joslyn on the New Stack Makers podcast, Drewitz discussed the role of AI and automation in achieving operational resiliency, especially in a context where teams are under pressure to be more productive.Automation, including generative AI code completion tools, is increasingly used to boost developer productivity. However, this may lead to shifting bottlenecks from developers to operations, creating new challenges. Drewitz emphasized the importance of considering the entire value chain and identifying areas where AI and automation can assist. For instance, automating repetitive tasks in incident response, such as checking APIs, closing ports, or database checks, can significantly reduce interruptions and productivity losses.PagerDuty's AI-powered platform leverages generative AI to automate tasks and create runbooks for incident handling, allowing engineers to focus on resolving root causes and restoring services. This includes drafting status updates and incident postmortem reports, streamlining incident response and saving time. Having an operations platform that can generate draft reports at the push of a button simplifies the process, making it easier to review and edit without starting from scratch.Learn more from The New Stack about AI, Automation, Incident Response, and PagerDuty:Operationalizing AI: Accelerating Automation, DataOps, AIOpsThree Ways Automation Can Improve Workplace CultureIncident Response: Three Ts to Rule Them AllFour Ways to Win Executive Buy-In for Automation

In this episode, Scott Johnston, CEO of Docker, highlights the evolving role of developers, emphasizing their increasing importance in architectural decision-making and tool development for applications. This shift in prioritizing a great developer experience and rapid tool development has led to substantial spending in the industry.Johnston expressed confidence that integrating generative AI into the developer experience will drive business growth and expand the customer base. He downplayed concerns about AI taking jobs, explaining that it would alleviate repetitive tasks, enabling developers to focus on more complex problem-solving. Johnston likened this evolution to expanding bike lanes in a city, leading to increased bike traffic, equating it to the development of more apps due to increased speed and efficiency.In his talk with TNS host, Alex Williams, Johnston emphasized that each advancement in programming languages and tools has expanded the developer market and driven greater demand for applications. Notably, the demand for over 750 million apps in the next two years, as reported by IDC, demonstrates the ever-increasing appetite for creative solutions from developers.Overall, Johnston sees the integration of generative AI and increasing development velocity as a multifaceted expansion that benefits developers and meets growing demand for applications in the market.Learn more from The New Stack about Generative AI and Docker:Generative AI News, Analysis, and ResourcesDocker Launches GenAI Stack and AI Assistant at DockerConDocker Rolls out 3 Tools to Speed and Ease Development

This episode of The New Stack Makers was recorded on the road at the Linux Foundation’s Open Source Summit Europe in Bilbao, Spain. A pair of technologists from Amazon Web Services (AWS) join us to discuss the development of Powertools for AWS Lambda. Andrea Amorosi, a senior solutions architect at AWS, and Leandro Damascena, a specialist solutions architect, share insights into how Powertools evolved from an observability tool to support more advanced use cases like ensuring workload safety, batch processing, streaming data, and idempotency.Powertools primarily supports Python, TypeScript, Java, and .NET. The latest feature, idempotency for TypeScript, was introduced to help customers achieve best practices for developing resilient and fault-tolerant workloads. By integrating these best practices during the development phase, Powertools reduces the need for costly re-architecting and rewriting of code.The success of Powertools can be attributed to its strong open source community, which fosters collaboration and contributions from users. AWS ensures transparency by conducting all project activities in the open, allowing anyone to understand and influence feature prioritization and contribute in various ways. Furthermore, the project's international support team offers assistance in multiple languages and time zones.A noteworthy aspect is that 40% of new Powertools features have been contributed by the community, providing contributors with valuable networking opportunities at a prominent tech giant like AWS. Overall, Powertools demonstrates how open source principles can thrive within a major corporation, offering benefits to both the company and the open source community.Learn more from The New Stack about Powertools, Lambda, and Amazon Web Services:AWS Offers a TypeScript Interface for Lambda ObservabilityHow Donating Open Source Code Can Advance Your CareerTurn AWS Lambda Functions Stateful with Amazon Elastic File System

KubeCon 2023 is set to feature three hot topics, according to Taylor Dolezal from the Cloud Native Computing Foundation. Firstly, GenAI and Large Language Models (LLMs) are taking the spotlight, particularly regarding their security and integration with legacy infrastructure. Platform engineering is also on the rise, with over 25 sessions at KubeCon Chicago focusing on its definition and how it benefits internal product teams by fostering a culture of product proliferation. Lastly, WebAssembly is emerging as a significant topic, with a dedicated day during the conference week. It is maturing and finding its place, potentially complementing containers, especially in edge computing scenarios. Wasm allows for efficient data processing before data reaches the cloud, adding depth to architectural possibilities.Overall, these three trends are expected to dominate discussions and presentations at KubeCon NA 2023, offering insights into the future of cloud-native technology.See what came out of the last KubeCon event in Amsterdam earlier this year:AI Talk at KubeConDon’t Force Containers and Disrupt WorkflowsA Boring Kubernetes Release

Digital.ai, an AI-powered DevSecOps platform, serves large enterprises such as financial institutions, insurance companies, and gaming firms. The primary challenge faced by these clients is scaling their DevOps practices across vast organizations. They aim to combine modern development methodologies like agile DevOps with the need for speed and intimacy with end-users on a large scale.This episode features a discussion between Wing To of Digital.ai and TNS host Heather Joslyn about platform engineering and the role of AI in enhancing automation. It delves into the dilemma of whether increased code production and release frequency driven by DevOps practices are inherently beneficial. Additionally, it explores the emerging challenge of AI-assisted development and how large enterprises are striving to realize productivity gains across their organizations.Digital.ai is focused on incorporating AI into automation to assist developers in creating and delivering code while helping organizations derive more business value from their software in production. The company employs templates to capture and replicate key aspects of software delivery processes and uses AI to automate the rapid setup of developer environments and tooling. These efforts contribute to the concept of the internal developer platform, which consists of multiple toolsets for tasks like creating pipelines and setting up various components.Learn more from The New Stack about Platform Engineering, DevSecOps and Digital.ai:Platform Engineering Overview, News, and TrendsSRE vs. DevOps vs. Platform EngineeringMeet the New DevSecOps

Moving workloads to the cloud presents cost prediction challenges. Traditional setups with on-premises hardware offer predictability, but cloud costs are usage-based and granular. In this podcast episode, Matt Stellpflug, a senior FinOps specialist at ProsperOps, discusses the complexities of forecasting cloud expenses with TNS host Heather Joslyn.Cloud users face fluctuating costs due to continuous deployments and changing workloads. There are additional expenses for data access and transfer. Stellpflug emphasizes the importance of establishing reference workloads and benchmarks for accurate forecasting.Engineers play a vital role in FinOps initiatives since they ensure application availability and system integrity. Stellpflug suggests collaborating with engineering teams to identify essential metrics. He co-authored an "Engineer's Guide to Cloud Cost Optimization," highlighting the distinction between resource and rate optimization. Best practices involve addressing high-impact, low-risk areas first, engaging subject matter experts for complex issues, and maintaining momentum. This episode also provides further insights into implementing FinOps for effective cloud cost management.Learn more from The New Stack about FinOps and ProsperOps:FinOps Overview, News, and TrendsProsperOps Wants to Automate Your FinOps StrategyEngineer’s Guide to Cloud Cost Optimization: Manual DIY OptimizationEngineer’s Guide to Cloud Cost Optimization: Engineering Resources in the CloudEngineer’s Guide to Cloud Cost Optimization: Prioritize Cloud Rate Optimization

In her keynote address at the Linux Foundation's Open Source Summit Europe, Fatima Sarah Khalid emphasized that being an ally is more than just superficial gestures like wearing pronouns on badges or correctly pronouncing coworkers' names. True allyship involves taking meaningful actions to support and uplift individuals from underrepresented or marginalized backgrounds. This support is essential, not only in obvious ways but also in everyday interactions, which collectively create a more inclusive community.Open source communities typically lack diversity, with only a small percentage of women, non-binary contributors, and individuals from underrepresented backgrounds. Khalid stressed the importance of improving diversity and inclusion through various means, including using inclusive language, facilitating asynchronous communication to accommodate global contributors, and welcoming non-technical contributions such as documentation.Khalid also provided insights on making open source events more inclusive, like welcoming newcomers and marginalized groups, providing quiet spaces and enforcing a code of conduct, and partnering newcomers with mentors. Moreover, she highlighted GitLab's unique approach to allyship within the organization, including the Ally Lab, which pairs employees from different backgrounds to learn about and understand each other's experiences.To encourage the audience to embrace allyship, Khalid shared a set of commitments to keep in mind, such as educating oneself about the experiences of marginalized groups, speaking up against inappropriate behavior, using one's voice to amplify marginalized voices, donating to support such groups, and advocating for equity and justice through social networks and connections. She also shared real-life examples of allyship, illustrating how meaningful actions can create positive change in communities.Khalid's discussion with host Jennifer Riggins emphasizes the significance of meaningful, everyday actions to promote allyship in open source communities and organizations, ultimately contributing to a more diverse, inclusive, and equitable tech industry.Learn more from The New Stack about Open Source, Allyship, and GitLab:Embracing Open Source for Greater Business ImpactLeadership and Inclusion in the Open Source CommunityHow Implicit Bias Impacts Open Source Diversity and InclusionInvesting in the Next Generation of Tech Talent

In a recent conversation at the Open Source Summit in Bilbao, Spain, Gabriel Colombo, the General Manager of the Linux Foundation Europe and the Executive Director of the Fintech Open Source Foundation, discussed the potential impact of the Cyber Resilience Act (CRA) on the open source community. The conversation shed light on the challenges and opportunities that the CRA presents to open source and how individuals and organizations can respond.The conversation began by addressing the Cyber Resilience Act and its significance. Gabriel Colombo explained that while the Act is being touted as a measure to bolster cybersecurity and national security, it could have unintended consequences for the open source ecosystem, particularly in Europe. The Act, currently in the legislative process, aims to address cybersecurity concerns but could inadvertently hinder open source development and collaboration.Jim Zemlin, the Executive Director of the Linux Foundation, had previously mentioned the importance of forks in open source development, emphasizing that they are a healthy aspect of the ecosystem. However, Colombo pointed out that the CRA could create a sense of unease, as it might deter people and companies from participating in open source projects or using open source software due to potential legal liabilities.To grasp the implications of the CRA, Colombo explained some of the key provisions. The initial drafts of the Act proposed potential liability for individual developers, open source foundations, and package managers. This raised concerns about the open source supply chain's potential vulnerability and the distribution of liability.As the Act evolves, the liability landscape has shifted somewhat. Individual developers may not be held liable unless they consistently receive donations from commercial companies. However, for open source foundations, especially those accepting recurring donations from commercial entities, there remains a concern about potential liabilities and the need to conform to the CRA's requirements.Colombo emphasized that this issue isn't limited to Europe. It could impact the entire global open source ecosystem and affect the ability of European developers and small to medium-sized businesses to participate effectively.The conversation highlighted the challenges open source communities face when engaging with policymakers. Open source is not structured like traditional corporations or industry consortiums, making it more challenging to present a unified front. Additionally, the legislative process can be slow and complex, which may not align with the rapid pace of technology development.The lack of proactive engagement from the European Commission and the absence of open source communities in the initial consultations on the Act are concerning. The understanding of open source, its nuances, and the role it plays in the broader software supply chain appears limited within policy-making circles.What Can Be Done?Gabriel Colombo stressed the importance of awareness and education. It is vital for individuals, businesses, and open source foundations to understand the implications of the CRA. The Linux Foundation and other organizations have launched campaigns to provide information and resources to help stakeholders comprehend the Act's potential impact.Being vocal and advocating for open source within your network, organization, and through public affairs channels can also make a difference. Engagement with policymakers, especially as the Act progresses through the legislative process, is crucial. Colombo encouraged businesses to emphasize the significance of open source in their operations and supply chains, making policymakers aware of how the CRA might affect their activities.In the face of the Cyber Resilience Act, the open source community must unite and actively engage with policymakers. It's essential to educate and raise awareness about the potential impact of the Act and advocate for a balanced approach that strengthens cybersecurity without stifling open source innovation.The Act's development is ongoing, and there is time for stakeholders to make their voices heard. With a united effort, the open source community can help shape the legislation to ensure that open source remains vibrant and resilient in the face of evolving cybersecurity challenges.Learn more from The New Stack about open source and Linux Foundation Europe:At Open Source Summit: Introducing Linux Foundation EuropeMaking Europe's 'Romantic' Open Source World More PracticalEmbracing Open Source for Greater Business Impact

In this episode of The New Stack Makers podcast, Uma Daniel, a product manager at UST, discusses the current complexities in the global economy, marked by low unemployment except in the tech industry, high inflation, high interest rates, a volatile stock market, and the looming threat of recession. Amid these challenges, organizations are seeking ways to enhance their operational efficiency.Daniel introduces the concept of FinOps, which goes beyond just managing cloud costs. Instead, it focuses on leveraging the cloud to generate revenue. This represents a cultural shift in many organizations, emphasizing the need for a mindset change across different departments, including business, finance, and procurement.She dispels misconceptions, such as the belief that only certain teams should be involved in the FinOps process. Daniel stresses that it's a collaborative effort involving various teams, and it's best to adopt FinOps at the beginning of a cloud journey. Once an organization is already established in the cloud, implementing FinOps becomes more challenging.To foster collaboration, Daniel suggests identifying team members willing to champion FinOps and forming cross-functional teams to lead the initiative. Regular committee meetings and the establishment of generic policies, such as project budgets, help control cloud spending.This episode, hosted by Heather Joslyn, provides insights into how to initiate and implement a FinOps strategy and highlights common ways in which organizations waste cloud resources.Learn more from The New Stack about FinOps and UST:Cloud Cost-Unit Economics — A Modern Profitability ModelWhat Is FinOps? Understanding FinOps Best Practices for CloudVery Large Enterprises Need a Different Approach to FinOps

Since the release of OpenAI's ChatGPT-3 in late 2022, various industries have been actively exploring its applications. Madhukar Kumar, CMO of SingleStore, discussed his experiments with large language models (LLMs) in this podcast episode with TNS host Heather Joslyn. He mentioned a specific LLM called Gorilla, which is trained on APIs and can generate APIs based on specific tasks. Kumar also talked about SingleStore Now, an AI conference, where they plan to teach attendees how to build generative AI applications from scratch, focusing on enterprise applications.Kumar highlighted a limitation with current LLMs - they are "frozen in time" and cannot provide real-time information. To address this, a method called "retrieval augmented generation" (RAG) has emerged. SingleStore is using RAG to keep LLMs updated. In this approach, a user query is first matched with up-to-date enterprise data to provide context, and then the LLM is tasked with generating answers based on this context. This method aims to prevent the generation of factually incorrect responses and relies on storing data as vectors for efficient real-time processing, which SingleStore enables.This strategy ensures that LLMs can provide current and contextually accurate information, making AI applications more reliable and responsive for enterprises.Learn more from The New Stack about LLMs and SingleStore:Top 5 Large Language Models and How to Use Them EffectivelyUsing ChatGPT for Questions Specific to Your Company Data6 Reasons Private LLMs Are Key for Enterprises

Observability in multi-cloud environments is becoming increasingly complex, as highlighted by Martin Mao, CEO and co-founder of Chronosphere. This challenge has two main components: a rise in customer-facing incidents, which demand significant engineering time for debugging, and the ineffectiveness and high cost of existing tools. These issues are creating a problematic return on investment for the industry.Mao discussed these observability challenges on The New Stack Makers podcast with host Heather Joslyn, emphasizing the need to help teams prioritize alerts and encouraging a shift left approach for security responsibility among developers. With the adoption of distributed cloud architectures, organizations are not only dealing with a surge in data but also facing a cultural shift towards DevOps, where developers are expected to be more accountable for their software in production.Historically, operations teams handled software in production, but in the cloud-native world, developers must take on these responsibilities themselves. Many current observability tools were designed for centralized operations teams, which creates a gap in addressing developer needs.Mao suggests that cloud-native observability tools should empower developers to run and maintain their software in production, providing insights into the complex environments they work in. Moreover, observability tools can assist developers in understanding the intricacies of their software, such as its dependencies and operational aspects.To streamline the data obtained from observability efforts and manage costs, Chronosphere introduced the "Observability Data Optimization Cycle." This framework starts with establishing centralized governance to set budgets for teams generating data. The goal is to optimize data usage to extract value without incurring unnecessary costs. This approach applies financial operations (FinOps) concepts to the observability space, helping organizations tackle the challenges of cloud-native observability.Learn more from The New Stack about Observability and Chronosphere:Observability Overview, News and Trends4 Key Observability Best PracticesTop Ways to Reduce Your Observability CostsTop 4 Factors for Cloud Native Observability Tool Selection

Platform engineering is gaining prominence due to the need for faster application deployment, which directly impacts business velocity. Valentina Alaria, Senior Director of Product at VMware, emphasizes that not all organizations pursuing platform engineering have the same goals, context, or pain points. They tailor solutions to each organization's specific needs. Some focus on rapid onboarding for junior developers, while others aim to reduce complexity, friction, and support larger development teams with fewer operational staff.Platform engineering aims to streamline collaboration between developers and operations engineers. Developers want portable code and the ability to focus on coding without worrying about production requirements. Operations engineers and platform teams seek a seamless environment for deploying applications in different contexts.Successful platform engineering initiatives involve strong collaboration models, fostering a cooperative approach rather than a siloed one. The goal is to create applications and value for the organization by facilitating effective interaction between developers and operations engineers.This podcast episode, hosted by Alex Williams of TNS, also delves into VMware Tanzu's latest tools for supporting platform engineering.Learn more from The New Stack about platform engineering and VMware Tanzu:Platform Engineering Overview, News and Trends6 Patterns for Platform Engineering SuccessA Guide to Open Source Platform EngineeringStreamline Platform Engineering with Kubernetes

ByConity is an open source project that emerged from ByteDance's use of Clickhouse, an open-source database system, to address their growing data volume. ByConity focuses on enhancing the separation of compute and storage, improving multitenancy support, and optimizing query performance in cloud-native environments.ByteDance's Vini Jaiswal, a principle developer advocate at the parent company of TikTok, highlights the power of open source in fostering innovation and collaboration. She shares her personal experience of leveraging open source to solve problems quickly and efficiently. She emphasizes the importance of getting involved in open source, even for those who might be hesitant, and suggests starting by identifying a pain point and making small contributions.ByConity's architecture, which separates compute and storage, offers benefits like preventing data lake corruption, read and write separation, elasticity, and scalability. Jaiswal also mentions her previous experience with open source during her time at CitiBank, where she realized how open source accelerated digital transformations.Throughout the conversation, Jaiswal underscores the strength of open source communities in collectively addressing challenges. She encourages listeners to embrace open source and start contributing, emphasizing how even small contributions can lead to significant impacts over time.The episode also delves into Jaiswal's involvement with other open source projects, such as PyTorch, and explores the intersection of open source and generative AI.Learn more from The New Stack about open source and cloud native environments:What Is 'Cloud Native' (and Why Does It Matter)?Cloud Native Ecosystem News and ResourcesHow to Build an Open Source Community

Along with discussing the emergence and ascension of platform engineering in this episode, we also discuss the role that Humanitec plays in helping organizations establish platforms for developers, as well as Backstage, a popular open source internal developer platform that was developed by Spotify for its own developers.An IDP, our guest Kaspar Von Grünberg explained, is a standardized interface for developers to build applications using a golden path of vetted tools and libraries, allowing for a high degree of efficiency for both the developers themselves as well as the engineers who are supporting the developers. They can include an integration and delivery plane, a continuous integration registry, a platform orchestrator, observability tools and a resource plane."How you're consuming this is a little bit up to the individual preference of the user, and what the platform team has configured for you. So we're seeing some teams like to use a user interface and some teams like to use code based interactions," Von Grünberg explained.In some ways, a IDP is reminiscent of the platform-as-a-service packages of a decade ago. They also were designed to help developer efficiency, though devs chafed at the limited number of tools they were allowed to use in these walled gardens. That was a mistake, Von Grünberg said.Those platforms required developers to use a small set of pre-defined times."We don't want to get back to those times, which is why we want to provide sensible defaults," Von Grünberg said. A good IDP will provide developers with "golden paths" or "paved roads" as Netflix calls them."Developers can stay on those paths if they want," Von Grünberg said. They can enjoy the security default and service-level agreements (SLAs) from the engineers. But developers are also free to leave the path and make low-level configurations on their own as well."Good platform engineering is never about covering all the use cases," he said.Learn more from The New Stack about platform engineering and Humanitec:Platform Engineering Overview, News, and TrendsHow to Pave Golden Paths That Actually Go SomewhereBuild Your IDP at Light Speed with a Platform Reference Architecture

In this episode of The New Stack Makers, technologist and author John Willis emphasized caution when considering AI solutions from vendors. He advised against blindly following vendor recommendations for "one-size-fits-all" AI products, likening it to discouraging learning Java in the past in favor of purchasing a product.Willis stressed that DevOps serves as an example of how human expertise, not just products, solves problems. He urged C-level executives to first understand AI's intricacies and then make informed purchasing decisions, suggesting a "DevOps redo" to encourage experimentation and collaboration, similar to the early days of the DevOps movement.Willis highlighted that early adopters of DevOps, like successful banks, heavily invested in developing their human capital. He cautioned against hasty product purchases, as the AI landscape is rife with startups that may quickly disappear or be acquired by larger companies.Instead, Willis advocated for educating teams on effective data management techniques, including retrieval augmentation, to fine-tune large language models. He emphasized the need for data cleansing to build robust data pipelines and prevent LLMs from generating undesirable code or sensitive information.According to Willis, the process becomes enjoyable when done correctly, especially for companies using LLMs at scale with retrieval augmentation. To ensure success, he suggested adding governance and structure, including content moderation and red-teaming of data, which vendors may not prioritize in their offerings.Learn more from The New Stack about DevOps and AI:AIOps: Is DevOps Ready for an Infusion of Artificial Intelligence?How to Build a DevOps Engineer in Just 6 MonthsPower up Your DevOps Workflow with AI and ChatGPT

Deliveroo, a prominent food delivery company, relies on Apache Flink, a distributed processing engine, to enhance its three-sided marketplace, connecting delivery drivers, restaurants, and customers. Seeking to improve real-time data streaming and gain insights into customer behavior, Deliveroo transitioned to Flink, comparing it to alternatives like Apache Spark and Kafka Streams. Flink, with feature parity to their previous platform, offered stability and scalability. They initially experimented with Flink on Kubernetes but turned to the Amazon Managed Service for Flink (MSF) for enhanced support and maintenance.Engineers from Deliveroo, Felix Angell and Duc Anh Khu, emphasized the need for flexibility in data modeling to accommodate their fast-paced product development. However, flexibility can be complex, often requiring data model adjustments. They expressed the desire for a self-serve configuration feature in MSF, allowing easy customization of low-level settings and auto-scaling based on application metrics. This move to Flink and MSF has empowered Deliveroo to focus on core responsibilities like continuous integration and delivery while efficiently managing their data processing needs.Learn more from The New Stack about Apache Flink and AWS:Kinesis, Kafka and Amazon Managed Service for Apache FlinkApache Flink for Real Time Data AnalysisApache Flink for Unbounded Data Streams

Over the past five to ten years, the testing of microservices has seen significant growth. This surge in testing can be attributed to the increasing adoption of microservices and Kubernetes, which signify a shift away from monolithic application architectures. Bruno Lopes, a leader at Kubernetes company incubator Kubeshop, noted this trend. Kubeshop has initiated six Kubernetes projects, including TestKube, a Kubernetes native testing framework led by Lopes.This rise in testing is making it more accessible to a wider audience and is enhancing the developer experience through automation. Developers now have more time to focus on innovation rather than manual testing. However, there is often a disconnect between development and testing, as developers move quickly, outpacing organizational adaptation to modern testing methods.Lopes emphasized the importance of testing before production deployment and advocated for creating production-resembling testing environments that allow for rapid deployment without waiting for manual tests. This approach is particularly critical for Site Reliability Engineering (SRE) teams who need to respond quickly to issues and minimize downtime for customers. In some cases, it's necessary to run tests within Kubernetes itself, a concept that may take time for companies to fully embrace as the developer experience continues to improve.Learn more from The New Stack about Kubernetes, Testing and TestKube:Testkube: A Cloud Native Testing Framework for KubernetesTop 5 Challenges in Modern Kubernetes TestingWhy You Should Start Testing in the Cloud Native Way

Apache Flink is an open-source framework and distributed processing engine designed for data analytics. It excels at handling tasks such as data joins, aggregations, and ETL (Extract, Transform, Load) operations. Moreover, it supports advanced real-time techniques like complex event processing.In this episode, Deepthi Mohan and Nagesh Honnalii from AWS discussed Apache Flink and the Amazon Managed Service for Apache Flink (MSF) with our host, Alex Williams. MSF is a service that caters to customers with varying infrastructure preferences. Some prefer complete control, while others want AWS to handle all infrastructure-related aspects.Use cases for MSF can be grouped into three categories. First, there's streaming ETL, which involves tasks like log aggregation for later auditing. Second, it supports real-time analytics, enabling customers to create dashboards for tasks like fraud detection. Third, it handles complex event processing, where data from multiple sources is joined and aggregated to extract meaningful insights.The origins of MSF trace back to the evolution of real-time data services within AWS. In 2013, AWS introduced Amazon Kinesis, while the open-source community developed Apache Kafka. These services paved the way for MSF by highlighting the need for real-time data processing.To provide more flexibility, AWS launched Kinesis Data Analytics in 2016, allowing customers to write code in JVM-based languages like Java and Scala. In 2018, AWS decided to incorporate Apache Flink into its Kinesis Data Analytics offering, leading to the birth of MSF.Today, thousands of customers use MSF, and AWS continues to enhance its offerings in the real-time data processing space, including the launch of Amazon MSK (Managed Streaming for Apache Kafka). To align with its foundation on Flink, AWS rebranded Kinesis Data Analytics for Apache Flink to Amazon Managed Service for Apache Flink, making it clearer for customers.Learn more from The New Stack about AWS and Apache Flink:Apache Flink for Real Time Data AnalysisApache Flink for Unbounded Data Streams3 Reasons Why You Need Apache Flink for Stream Processing

Modern developer conferences like the upcoming Infobip Shift Conference in Croatia are centered around themes. At this particular event for developers, you can expect a lot of focus to be on the developer experience and artificial intelligence (AI).Ivan Burazin, Chief Development Experience Officer at InfoBip, joined us on the show and emphasizes that developers spend a substantial portion of their time not coding, often losing 50 to 70% of their productive hours to non-coding activities, such as setting up environments, running tests, and building code. This highlights the importance of improving the developer experience to enhance productivity.The developer experience has both internal and external dimensions. Externally, it impacts customer experience, while internally, it influences development velocity. A better developer experience translates to faster and more efficient coding.The Shift Conference will feature talks on six stages, one of which will focus on the developer experience, addressing its internal and external aspects. Additionally, AI will take center stage at another segment of the conference.Although there may not be an abundance of true AI experts taking the stage, the focus will be on how individuals and companies can leverage AI to create products and services. It's recognized that AI will play a pivotal role in the future of every industry, and the conference aims to explore practical applications and strategies for integrating AI into various businesses.Overall, the Shift Conference aims to address the challenges developers face in optimizing their productivity and explore the growing importance of AI in shaping the future of businesses and products.Learn more from The New Stack about the developer experience and InfoBip Shift:7 Principles and 10 Tactics to Make You a 10x DeveloperThe Challenges of Marketing Software Tools to DevelopersA Guide to Better Developer Experience

This episode delves into Apache Flink, a versatile platform for executing both batch and real-time streaming data analysis tasks. This session marks the beginning of a three-part series unveiling Amazon Web Services' (AWS) new managed service built on Flink. Future episodes will explore this service in detail and examine customer experiences.The podcast features insights from Danny Cranmer, a principal engineer at AWS and an Apache Flink PMC and Committer, along with Hong Teoh, a software development engineer at AWS.Flink stands out as a high-level framework for defining data analytics jobs, accommodating both batch and streaming data sets. It offers APIs for building analysis jobs in various languages, including Java, Python, and SQL. Flink also provides a distributed job execution engine with fault tolerance and horizontal scaling capabilities.One prominent use case is Extract-Transform-Load (ETL), where raw data is swiftly processed for specific workloads. Flink excels in delivering low-latency transformations for unbounded data streams. Additionally, Flink supports event-driven applications, responding immediately to triggers such as user requests for weather data.Flink ensures exactly-once processing, critical for scenarios like financial transactions. It employs checkpoints to maintain data integrity in case of node failures.The podcast also touches on AWS's role in supporting the open-source Flink project and the future outlook for this powerful data processing framework.Learn more from The New Stack about Apache Flink:3 Reasons Why You Need Apache Flink for Stream ProcessingApache Flink for Unbounded Data Streams8 Real-Time Data Best Practices

In an interview with The New Stack, renowned technologist Adrian Cockcroft discussed the process of fine-tuning Large Language Models (LLMs) through prompt engineering. Cockcroft, known for his roles at Netflix and Amazon Web Services, explained how to obtain tailored programming advice from an LLM. By crafting specific prompts like asking the model to provide code in the style of a certain expert programmer, such as Java's James Gosling, users can guide the AI's output.Prompt engineering involves setting up conversations to bias the AI's responses. These prompts are becoming more advanced with plugins and loaded information that shape the model's behavior before use. Cockcroft highlighted the concept of fine-tuning, where models are adapted beyond what a prompt can contain. Companies are incorporating vast amounts of their internal data, like wiki pages and corporate documents, to train the model to understand their specific domain and processes.Cockcroft pointed out the efficacy of ChatGPT within certain tasks, illustrated by his experience using it for data analysis and programming assistance. He also discussed the growing need for improved results from LLMs, which has led to the demand for vector databases. These databases store word meanings as vectors with associated weights, enabling fuzzy matching for enhanced information retrieval from LLMs. In essence, Cockcroft emphasized the multifaceted process of shaping and optimizing LLMs through prompt engineering and fine-tuning, reflecting the evolving landscape of AI-human interactions.Learn more from The New Stack about LLMs and Prompt Engineering:Top 5 Large Language Models and How to Use Them EffectivelyThe Pros (And Con) of Customizing Large Language ModelsPrompt Engineering: Get LLMs to Generate the Content You WantDeveloper Tips in AI Prompt Engineering

TechWorld with Nana is one of the most popular resources for people looking to get into or progress a DevOps career. Nana Janashia, the creator of TechWorld with Nana, is a DevOps trainer and consultant who joined us to discuss why DevOps is needed now more than ever and how this is the perfect time to begin a career in DevOps.Host Alex Williams and Nana go over the key concepts of DevOps. Then they talk about how the complexity of tools can sidetrack and complicate the learning process for those new to DevOps and why focusing on concepts rather than tools the way to go. Before wrapping up the conversation, they even talk about the best ways for people to get involved who are new to DevOps.Nana's journey into DevOps commenced during her time as an engineer in Austria, where she began exploring Kubernetes. As inquiries from colleagues poured in, she recognized her knack for demystifying complex topics, catalyzing her passion for teaching. Viewers attest to switching to DevOps careers after watching her videos.Throughout the conversation, we learned how people can discover the world of DevOps through TechWorld with Nana as an expert guide. With a large YouTube audience, online courses, workshops, and corporate training, Nana has empowered countless individuals in advancing their DevOps expertise. The six-month boot camps from TechWorld with Nana encompass a comprehensive curriculum, starting with fundamentals and culminating in hands-on programming abilities, Python automation, configuration management, and Prometheus-based monitoring.Nana underscores that DevOps, still a relatively nascent profession, suffers from role ambiguity both among engineers and within companies aspiring to implement it. This confusion stems from differing workflows and environments when engineers switch jobs. Nana's insights bring clarity to these challenges, acknowledging the evolving chaos of the DevOps culture and its driving force for innovation in managing intricate distributed technologies.Learn more about DevOps from TNS, Roadmap (our sister site), and TechWorld with Nana:TechWorld with Nana - DevOps BootcampTechWorld with Nana - DevSecOps BootcampDevOps Learning RoadmapDevOps News, Trends, and Analysis

Explore the complex intersection of AI and open source with insights from experts in this illuminating discussion. Amanda Brock, CEO of OpenUK, reveals the challenges in labeling AI as open source amidst legal ambiguities. The dialogue, led by TNS host Alex Williams, delves into the evolution of open source licensing, its departure from traditional models, and the complications arising from applying open source principles to AI, which encompasses sensitive data governed by privacy laws.The focus turns to "Llama 2," a contentious example where Meta labeled their language model as open source, sparking confusion. Notable guests Erica Brescia, Managing Director at Redpoint Ventures, and Steven Vaughan-Nichols, founder of Open Source Watch, weigh in on this topic. Brock emphasizes that AI's complexity prevents it from aligning with the Open Source Definition, necessitating a clear distinction between open innovation and open source.Amidst these debates, the Open Source Initiative (OSI) is crafting a new definition tailored for AI, sparking anticipation and discussion about its implications. The necessity for an evolved understanding of open source and its licenses is underscored, as the rapid evolution of technology challenges established norms. The journey concludes with reflections on vendors transitioning from open source licenses to Server Side Public License (SSPL) due to cloud-related considerations, raising questions about the future of open source in a dynamically changing tech landscape.Learn more from The New Stack about open source and AI:Open Source May Yet Eat Google's and OpenAI's AI LunchOpen Source Movement Emerging in AI To Counter GreedHow AI Can Learn from the Struggles of Open Source

Discover how large language models and generative AI are revolutionizing DevOps with PromptOps. The company, initially known as CtrlStack, introduces its unique process engine that comprehends human requests, reads knowledge bases, and generates code on the fly to accomplish tasks. Dev Nag, the CEO, explains how PromptOps saves users time and money by automating routine operations in this podcast episode with The New Stack.Dev Nag is joined by GK Brar, PromptOps' founding engineer, and our host Joab Jackson as they delve into the concept of generative AI and its potential benefits for DevOps. Traditionally, DevOps tasks often involve repetitive troubleshooting and reporting, making automation essential. PromptOps specializes in intent matching, understanding nuanced requests and providing the right solutions.Notably, PromptOps employs generative AI offline to prepare for automating common actions and enhancing the user experience. Unlike others, PromptOps aims beyond simple enhancements. It aspires to transform the entire DevOps landscape by leveraging this groundbreaking technology.Tune in to the podcast to gain deeper insights into this transformative approach that PromptOps brings to DevOps thanks to the power and possibilities of generative AI.Learn more from The New Stack about DevOps and PromptOps:DevOps News, Trends, Analysis and ResourcesHow to Use ChatGPT for IT Security AuditWhat We Learned from Building a Chatbot