Using openSUSE Leap as base layer operating system for AI/ML container orchestration using SUSE Rancher. (osc25)

The exponentially growing **AI/ML** and **LLMs** advancements bring concerns about privacy, as there is a risk of data exposure to online LLMs service providers. Setting up **LLMs in-house** requires a **high computational cost** which is a major obstacle for businesses across various sectors such as Retail, Healthcare, Finance, etc. These industries seek to leverage the power of LLMs to drive **profitability** in their overall business while maintaining **control over their data.** In this session, we will explore the **Edge Ecosystem Analytics** and its transformative potential in **GenAI Applications** through seamless orchestration via **openSUSE Leap** and **Rancher-managed Kubernetes**. This approach helps overcome challenges in adopting and deploying cutting-edge GenAI applications securely and efficiently at the edge. Key Topics: - Overview of **Large Language Models (LLMs)** - Scope for **Edge Computing** in AI revolution - Benefits over privacy concerns by **localization of LLMs** - Real-world Application Showcase by leveraging **GenAI for Edge Ecosystem Analytics** - Integration of Retrieval-Augmented Generation (**RAG**) Pipeline into **Rancher & K3s** - Challenges while deploying **GenAI applications at the Edge** This short talk will showcase a real-world GenAI-based application, highlighting the utilization of the **RAG pipeline** as well as a **data modeling pipeline** to continually improve analytic outputs and its seamless integration with **Rancher and K3s**. Attendees will learn about **Rancher and K3s** in managing Kubernetes deployments for GenAI applications, LLM optimization techniques such as **RAG**, overview of **Fine Tuning** and **AI Agents**. Licensed to the public under https://creativecommons.org/licenses/by-sa/4.0/ about this event: https://c3voc.de