Pre-training LLMs: One Model To Rule Them All? with Talfan Evans, DeepMind
Thinking Machines: AI & Philosophy
May 18, 202437m 36s
Audio is streamed directly from the publisher (media.transistor.fm) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
Talfan Evans is a research engineer at DeepMind, where he focuses on data curation and foundational research for pre-training LLMs and multimodal models like Gemini. I ask Talfan:
- Will one model rule them all?
- What does "high quality data" actually mean in the context of LLM training?
- Is language model pre-training becoming commoditized?
- Are companies like Google and OpenAI keeping their AI secrets to themselves?
- Does the startup or open source community stand a chance next to the giants?
Also check out Talfan's latest paper at DeepMind, Bad Students Make Good Teachers.
Topics
machine learningartificial intelligencemlopsMLAI