[Linkpost] “The Scaling Paradox” by Toby_Ord

This is a link post.<p> AI capabilities have improved remarkably quickly, fuelled by the explosive scale-up of resources being used to train the leading models. But if you examine the scaling laws that inspired this rush, they actually show extremely poor returns to scale. What's going on?</p><p> AI Scaling is Shockingly Impressive</p><p> The era of LLMs has seen remarkable improvements in AI capabilities over a very short time. This is often attributed to the AI scaling laws — statistical relationships which govern how AI capabilities improve with more parameters, compute, or data. Indeed AI thought-leaders such as Ilya Sutskever and Dario Amodei have said that the discovery of these laws led them to the current paradigm of rapid AI progress via a dizzying increase in the size of frontier systems.</p><p> Before the 2020s, most AI researchers were looking for architectural changes to push the frontiers of AI forwards. The idea that scale alone was sufficient to provide the entire range of faculties involved in intelligent thought was unfashionable and seen as simplistic.</p><p> A key reason it worked was the tremendous versatility of text. As Turing had noted more than 60 years earlier, almost any challenge that one could pose to [...]</p> <p>---</p> <p><b>First published:</b><br/> January 30th, 2026 </p> <p><b>Source:</b><br/> <a href="https://forum.effectivealtruism.org/posts/742xJNTqer2Dt9Cxx/the-scaling-paradox?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://forum.effectivealtruism.org/posts/742xJNTqer2Dt9Cxx/the-scaling-paradox</a> </p> <p><strong>Linkpost URL:</strong><br><a href="https://forum.effectivealtruism.org/out?url=https%3A%2F%2Fwww.tobyord.com%2Fwriting%2Fthe-scaling-paradox" rel="noopener noreferrer" target="_blank">https://www.tobyord.com/writing/the-scaling-paradox</a></p> <p>---</p> <p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=ea_forum&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p> <p>---</p><div style="max-width: 100%";><p><strong>Images from the article:</strong></p><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/wkuwqiigcsbqwcgpq23u" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/wkuwqiigcsbqwcgpq23u" alt="Two scatter plots showing "o1 AIME accuracy during training" and "o1 AIME accuracy at test time" versus compute on log scale." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/ynpe4iioo8di0wpqj6gp" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/ynpe4iioo8di0wpqj6gp" alt="Three graphs showing relationships between test loss and compute, dataset size, and parameters with power law scaling equations." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/shvvsuwofncaqx4stpj3" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/shvvsuwofncaqx4stpj3" alt="Graph showing compute required versus accuracy, with exponential growth curve." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/vee5a9ttz0tfvuh8ieiz" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/vee5a9ttz0tfvuh8ieiz" alt="Graph showing training loss versus FLOPS for different model parameter sizes." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/fqtruxinmtnrcmtdieb6" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/fqtruxinmtnrcmtdieb6" alt="Clock icon equals bicycle icon, two hundred." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/eyag74oarvxme3asspyo" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/eyag74oarvxme3asspyo" alt="Dog looking at camera with blurred person standing nearby." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/yvtyx1ciq6ndwlbwjdij" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/yvtyx1ciq6ndwlbwjdij" alt="Mathematical equation showing D equals k over p times d to the power of n." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/lmqgqjf6mlotffaot1et" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/742xJNTqer2Dt9Cxx/lmqgqjf6mlotffaot1et" alt="Mathematical equation showing N equals k divided by N times a to the power of 13." style="max-width: 100%;" /></a><p><em>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</em></p></div>