Season 2 · Episode 1700
Can LLMs Learn Continuously Without Forgetting?
We explore a new approach: micro-training updates every few days to keep AI knowledge fresh without constant web searches.
My Weird Prompts · Daniel Rosehill
March 29, 202621m 24s
Audio is streamed directly from the publisher (dts.podtrac.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
Retrieval-Augmented Generation (RAG) is standard for current AI, but it adds latency and complexity. This episode explores an alternative: micro-training LLMs to embed recent knowledge directly into their weights. We discuss the technical feasibility, the risk of catastrophic forgetting, and how LoRA adapters might solve the "goldfish memory" problem. Learn why this approach could be a game-changer for autonomous agents, despite the risks of data poisoning and the need for a "digital editor-in-chief."