Season 1 · Episode 56
Building an AI Model from Scratch: The Hidden Costs
Building an AI model from scratch? It's a brutal reality of trillions of tokens and millions in GPUs. Discover the hidden costs of modern AI.
My Weird Prompts · Daniel Rosehill
December 11, 202528m 55s
Audio is streamed directly from the publisher (dts.podtrac.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
What would it actually take to build a large language model completely from scratch? Corn and Herman break down the brutal reality: from data collection across trillions of tokens to GPU clusters costing millions, they explore why almost nobody does this anymore. This thought experiment reveals every layer of modern AI development, the astronomical expenses involved, and why fine-tuning existing models makes so much more sense. A deep dive into the machinery behind ChatGPT and Claude.