Doom Debates!

Devin Elliot is a former pro snowboarder turned software engineer who has logged thousands of hours building AI systems. His P(Doom) is a flat ⚫. He argues that worrying about an AI takeover is as irrational as fearing your car will sprout wings and fly away.We spar over the hard limits of current models: Devin insists LLMs are hitting a wall, relying entirely on external software “wrappers” to feign intelligence. I push back, arguing that raw models are already demonstrating native reasoning and algorithmic capabilities.Devin also argues for decentralization by claiming that nuclear proliferation is safer than centralized control.We end on a massive timeline split: I see superintelligence in a decade, while he believes we’re a thousand years away from being able to “grow” computers that are truly intelligence.Timestamps00:00:00 Episode Preview00:01:03 Intro: Snowboarder to Coder00:03:30 "I Do Not Have a P(Doom)"00:06:47 Nuclear Proliferation & Centralized Control00:10:11 The "Spotify Quality" House Analogy00:17:15 Ideal Geopolitics: Decentralized Power00:25:22 Why AI Can't "Fly Away"00:28:20 The Long Addition Test: Native or Tool?00:38:26 Is Non-Determinism a Feature or a Bug?00:52:01 The Impossibility of Mind Uploading00:57:46 "Growing" Computers from Cells01:02:52 Timelines: 10 Years vs. 1,000 Years01:11:40 "Plastic Bag Ghosts" & Builder Intuition01:13:17 Summary of the Debate01:15:30 Closing ThoughtsLinksDevin’s Twitter — https://x.com/devinjelliot---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Michael Timothy Bennett, Ph.D, is an award-winning young researcher who has developed a new formal framework for understanding intelligence. He has a TINY P(Doom) because he claims superintelligence will be resource-constrained and tend toward cooperation.In this lively debate, I stress-test Michael’s framework and debate whether its theorized constraints will actually hold back superintelligent AI.Timestamps* 00:00 Trailer* 01:41 Introducing Michael Timothy Bennett* 04:33 What’s Your P(Doom)?™* 10:51 Michael’s Thesis on Intelligence: “Abstraction Layers”, “Adaptation”, “Resource Efficiency”* 25:36 Debate: Is Einstein Smarter Than a Rock?* 39:07 “Embodiment”: Michael’s Unconventional Computation Theory vs Standard Computation* 48:28 “W-Maxing”: Michael’s Intelligence Framework vs. a Goal-Oriented Framework* 59:47 Debating AI Doom* 1:09:49 Debating Instrumental Convergence* 1:24:00 Where Do You Get Off The Doom Train™ — Identifying The Cruxes of Disagreement* 1:44:13 Debating AGI Timelines* 1:49:10 Final RecapLinksMichael’s website — https://michaeltimothybennett.comMichael’s Twitter — https://x.com/MiTiBennettMichael’s latest paper, “How To Build Conscious Machines” — https://osf.io/preprints/thesiscommons/wehmg_v1?view_onlyDoom Debates' Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

My guest today achieved something EXTREMELY rare and impressive: Coming onto my show with an AI optimist position, then admitting he hadn’t thought of my counterarguments before, and updating his beliefs in realtime! Also, he won the 2013 Nobel Prize in computational biology.I’m thrilled that Prof. Levitt understands the value of raising awareness about imminent extinction risk from superintelligent AI, and the value of debate as a tool to uncover the truth — the dual missions of Doom Debates!Timestamps0:00 — Trailer1:18 — Introducing Michael Levitt4:20 — The Evolution of Computing and AI12:42 — Measuring Intelligence: Humans vs. AI23:11 — The AI Doom Argument: Steering the Future25:01 — Optimism, Pessimism, and Other Existential Risks34:15 — What’s Your P(Doom)™36:16 — Warning Shots and Global Regulation55:28 — Comparing AI Risk to Pandemics and Nuclear War1:01:49 — Wrap-Up1:06:11 — Outro + New AI safety resourceShow NotesMichael Levitt’s Twitter — https://x.com/MLevitt_NP2013-- Get full access to Doom Debates at lironshapira.substack.com/subscribe

Michael Ellsberg, son of the legendary Pentagon Papers leaker Daniel Ellsberg, joins me to discuss the chilling parallels between his father’s nuclear war warnings and today’s race to AGI.We discuss Michael’s 99% probability of doom, his personal experience being “obsoleted” by AI, and the urgent moral duty for insiders to blow the whistle on AI’s outsize risks.Timestamps0:00 Intro1:29 Introducing Michael Ellsberg, His Father Daniel Ellsberg, and The Pentagon Papers5:49 Vietnam War Parallels to AI: Lies and Escalation25:23 The Doomsday Machine & Nuclear Insanity48:49 Mutually Assured Destruction vs. Superintelligence Risk55:10 Evolutionary Dynamics: Replicators and the End of the “Dream Time”1:10:17 What’s Your P(doom)?™1:14:49 Debating P(Doom) Disagreements1:26:18 AI Unemployment Doom1:39:14 Doom Psychology: How to Cope with Existential Risk1:50:56 The “Joyless Singularity”: Aligned AI Might Still Freeze Humanity2:09:00 A Call to Action for AI InsidersShow Notes:Michael Ellsberg’s website — https://www.ellsberg.com/Michael’s Twitter — https://x.com/MichaelEllsbergDaniel Ellsberg’s website — https://www.ellsberg.net/The upcoming book, “Truth and Consequence” — https://geni.us/truthandconsequenceMichael’s AI-related substack “Mammalian Wetware” — https://mammalianwetware.substack.com/Daniel’s debate with Bill Kristol in the run-up to the Iraq war — https://www.youtube.com/watch?v=HyvsDR3xnAg--Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Today's Debate: Should we ban the development of artificial superintelligence until scientists agree it is safe and controllable?Arguing FOR banning superintelligence until there’s a scientific consensus that it’ll be done safely and controllably and with strong public buy-in: Max Tegmark. He is an MIT professor, bestselling author, and co-founder of the Future of Life Institute whose research has focused on artificial intelligence for the past 8 years.Arguing AGAINST banning superintelligent AI development: Dean Ball. He is a Senior Fellow at the Foundation for American Innovation who served as a Senior Policy Advisor at the White House Office of Science and Technology Policy under President Trump, where he helped craft America’s AI Action Plan.Two of the leading voices on AI policy engaged in high-quality, high-stakes debate for the benefit of the public!This is why I got into the podcast game — because I believe debate is an essential tool for humanity to reckon with the creation of superhuman thinking machines.Timestamps0:00 - Episode Preview1:41 - Introducing The Debate3:38 - Max Tegmark’s Opening Statement5:20 - Dean Ball’s Opening Statement9:01 - Designing an “FDA for AI” and Safety Standards21:10 - Liability, Tail Risk, and Biosecurity29:11 - Incremental Regulation, Timelines, and AI Capabilities54:01 - Max’s Nightmare Scenario57:36 - The Risks of Recursive Self‑Improvement1:08:24 - What’s Your P(Doom)?™1:13:42 - National Security, China, and the AI Race1:32:35 - Closing Statements1:44:00 - Post‑Debate Recap and Call to ActionShow NotesStatement on Superintelligence released by Max’s organization, the Future of Life Institute — https://superintelligence-statement.org/Dean’s reaction to the Statement on Superintelligence — https://x.com/deanwball/status/1980975802570174831America’s AI Action Plan — https://www.whitehouse.gov/articles/2025/07/white-house-unveils-americas-ai-action-plan/“A Definition of AGI” by Dan Hendrycks, Max Tegmark, et. al. —https://www.agidefinition.ai/Max Tegmark’s Twitter — https://x.com/tegmarkDean Ball’s Twitter — https://x.com/deanwball Get full access to Doom Debates at lironshapira.substack.com/subscribe

Max Harms and Jeremy Gillen are current and former MIRI researchers who both see superintelligent AI as an imminent extinction threat.But they disagree on whether it’s worthwhile to try to aim for obedient, “corrigible” AI as a singular target for current alignment efforts.Max thinks corrigibility is the most plausible path to build ASI without losing control and dying, while Jeremy is skeptical that this research path will yield better superintelligent AI behavior on a sufficiently early try.By listening to this debate, you’ll find out if AI corrigibility is a relatively promising effort that might prevent imminent human extinction, or an over-optimistic pipe dream.Timestamps0:00 — Episode Preview1:18 — Debate Kickoff3:22 — What is Corrigibility?9:57 — Why Corrigibility Matters11:41 — What’s Your P(Doom)™16:10 — Max’s Case for Corrigibility19:28 — Jeremy’s Case Against Corrigibility21:57 — Max’s Mainline AI Scenario28:51 — 4 Strategies: Alignment, Control, Corrigibility, Don’t Build It37:00 — Corrigibility vs HHH (”Helpful, Harmless, Honest”)44:43 — Asimov’s 3 Laws of Robotics52:05 — Is Corrigibility a Coherent Concept?1:03:32 — Corrigibility vs Shutdown-ability1:09:59 — CAST: Corrigibility as Singular Target, Near Misses, Iterations1:20:18 — Debating if Max is Over-Optimistic1:34:06 — Debating if Corrigibility is the Best Target1:38:57 — Would Max Work for Anthropic?1:41:26 — Max’s Modest Hopes1:58:00 — Max’s New Book: Red Heart2:16:08 — OutroShow NotesMax’s book Red Heart — https://www.amazon.com/Red-Heart-Max-Harms/dp/108822119XLearn more about CAST: Corrigibility as Singular Target — https://www.lesswrong.com/s/KfCjeconYRdFbMxsy/p/NQK8KHSrZRF5erTbaMax’s Twitter — https://x.com/raelifinJeremy’s Twitter — https://x.com/jeremygillen1---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Holly Elmore leads protests against frontier AI labs, and that work has strained some of her closest relationships in the AI-safety community.She says AI safety insiders care more about their reputation in tech circles than actually lowering AI x-risk. This is our full conversation from my “If Anyone Builds It, Everyone Dies” unofficial launch party livestream on Sept 16, 2025.Timestamps0:00 Intro1:06 Holly’s Background and The Current Activities of PauseAI US4:41 The Circular Firing Squad Problem of AI Safety7:23 Why the AI Safety Community Resists Public Advocacy11:37 Breaking with Former Allies at AI Labs13:00 LessWrong’s reaction to Eliezer’s public turnShow NotesPauseAI US — https://pauseai-us.orgInternational PauseAI — https://pauseai.infoHolly’s Twitter — https://x.com/ilex_ulmusHolly’s Substack: https://substack.com/@hollyelmoreHolly’s post covering how AI isn’t another “technology”: https://hollyelmore.substack.com/p/the-technology-bucket-errorRelated EpisodesHolly and I dive into the rationalist community’s failure to rally behind a cause: https://lironshapira.substack.com/p/lesswrong-circular-firing-squadThe full IABED livestream: https://lironshapira.substack.com/p/if-anyone-builds-it-everyone-dies-party Get full access to Doom Debates at lironshapira.substack.com/subscribe

Sparks fly in the finale of my series with ex-MIRI researcher Tsvi Benson-Tilsen as we debate his AGI timelines.Tsvi is a champion of using germline engineering to create smarter humans who can solve AI alignment.I support the approach, even though I’m skeptical it’ll gain much traction before AGI arrives.Timestamps0:00 Debate Preview0:57 Tsvi’s AGI Timeline Prediction 3:03 The Least Impressive Task AI Cannot Do In 2 years6:13 Proposed Task: Solve Cantor’s Theorem From Scratch 8:20 AI Has Limitations Related to Sample Complexity 11:41 We Need Clear Goalposts for Better AGI Predictions 13:19 Counterargument: LLMs May Not Be a Path to AGI16:01 Is Tsvi Setting a High Bar for Progress Towards AGI? 19:17 AI Models Are Missing A Spark of Creativity28:17 Liron’s “Black Box” AGI Test 32:09 Are We Going to Enter an AI Winter? 35:09 Who Is Being Overconfident? 42:11 If AI Makes Progress on Benchmarks, Would Tsvi Shorten His Timeline? 50:34 Recap & Tsvi’s ResearchShow NotesLearn more about Tsvi’s organization, the Berkeley Genomics Project — https://berkeleygenomics.orgDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Today I’m sharing my AI doom interview on Donal O’Doherty’s podcast.I lay out the case for having a 50% p(doom). Then Donal plays devil’s advocate and tees up every major objection the accelerationists throw at doomers. See if the anti-doom arguments hold up, or if the AI boosters are just serving sophisticated cope.Timestamps0:00 — Introduction & Liron’s Background 1:29 — Liron’s Worldview: 50% Chance of AI Annihilation 4:03 — Rationalists, Effective Altruists, & AI Developers 5:49 — Major Sources of AI Risk 8:25 — The Alignment Problem 10:08 — AGI Timelines 16:37 — Will We Face an Intelligence Explosion? 29:29 — Debunking AI Doom Counterarguments 1:03:16 — Regulation, Policy, and Surviving The Future With AIShow NotesIf you liked this episode, subscribe to the Collective Wisdom Podcast for more deeply researched AI interviews: https://www.youtube.com/@DonalODoherty Transcript Introduction & Liron’s BackgroundDonal O’Doherty 00:00:00Today I’m speaking with Liron Shapira. Liron is an investor, he’s an entrepreneur, he’s a rationalist, and he also has a popular podcast called Doom Debates, where he debates some of the greatest minds from different fields on the potential of AI risk.Liron considers himself a doomer, which means he worries that artificial intelligence, if it gets to superintelligence level, could threaten the integrity of the world and the human species.Donal 00:00:24Enjoy my conversation with Liron Shapira.Donal 00:00:30Liron, welcome. So let’s just begin. Will you tell us a little bit about yourself and your background, please? I will have introduced you, but I just want everyone to know a bit about you.Liron Shapira 00:00:39Hey, I’m Liron Shapira. I’m the host of Doom Debates, which is a YouTube show and podcast where I bring in luminaries on all sides of the AI doom argument.Liron 00:00:49People who think we are doomed, people who think we’re not doomed, and we hash it out. We try to figure out whether we’re doomed. I myself am a longtime AI doomer. I started reading Yudkowsky in 2007, so it’s been 18 years for me being worried about doom from artificial intelligence.My background is I’m a computer science bachelor’s from UC Berkeley.Liron 00:01:10I’ve worked as a software engineer and an entrepreneur. I’ve done a Y Combinator startup, so I love tech. I’m deep in tech. I’m deep in computer science, and I’m deep into believing the AI doom argument.I don’t see how we’re going to survive building superintelligent AI. And so I’m happy to talk to anybody who will listen. So thank you for having me on, Donal.Donal 00:01:27It’s an absolute pleasure.Liron’s Worldview: 50% Chance of AI AnnihilationDonal 00:01:29Okay, so a lot of people where I come from won’t be familiar with doomism or what a doomer is. So will you just talk through, and I’m very interested in this for personal reasons as well, your epistemic and philosophical inspirations here. How did you reach these conclusions?Liron 00:01:45So I often call myself a Yudkowskian, in reference to Eliezer Yudkowsky, because I agree with 95% of what he writes, the Less Wrong corpus. I don’t expect everybody to get up to speed with it because it really takes a thousand hours to absorb it all.I don’t think that it’s essential to spend those thousand hours.Liron 00:02:02I think that it is something that you can get in a soundbite, not a soundbite, but in a one-hour long interview or whatever. So yeah, I think you mentioned epistemic roots or whatever, right? So I am a Bayesian, meaning I think you can put probabilities on things the way prediction markets are doing.Liron 00:02:16You know, they ask, oh, what’s the chance that this war is going to end? Or this war is going to start, right? What’s the chance that this is going to happen in this sports game? And some people will tell you, you can’t reason like that.Whereas prediction markets are like, well, the market says there’s a 70% chance, and what do you know? It happens 70% of the time. So is that what you’re getting at when you talk about my epistemics?Donal 00:02:35Yeah, exactly. Yeah. And I guess I’m very curious as well about, so what Yudkowsky does is he conducts thought experiments. Because obviously some things can’t be tested, we know they might be true, but they can’t be tested in experiments.Donal 00:02:49So I’m just curious about the role of philosophical thought experiments or maybe trans-science approaches, in terms of testing questions that we can’t actually conduct experiments on.Liron 00:03:00Oh, got it. Yeah. I mean this idea of what can and can’t be tested. I mean, tests are nice, but they’re not the only way to do science and to do productive reasoning.Liron 00:03:10There are times when you just have to do your best without a perfect test. You know, a recent example was the James Webb Space Telescope, right? It’s the successor to the Hubble Space Telescope. It worked really well, but it had to get into this really difficult orbit.This very interesting Lagrange point, I think in t

Tsvi Benson-Tilsen spent seven years tackling the alignment problem at the Machine Intelligence Research Institute (MIRI). Now he delivers a sobering verdict: humanity has made “basically 0%” progress towards solving it. Tsvi unpacks foundational MIRI research insights like timeless decision theory and corrigibility, which expose just how little humanity actually knows about controlling superintelligence. These theoretical alignment concepts help us peer into the future, revealing the non-obvious, structural laws of “intellidynamics” that will ultimately determine our fate. Time to learn some of MIRI’s greatest hits.P.S. I also have a separate interview with Tsvi about his research into human augmentation: Watch here!Timestamps 0:00 — Episode Highlights 0:49 — Humanity Has Made 0% Progress on AI Alignment 1:56 — MIRI’s Greatest Hits: Reflective Probability Theory, Logical Uncertainty, Reflective Stability 6:56 — Why Superintelligence is So Hard to Align: Self-Modification 8:54 — AI Will Become a Utility Maximizer (Reflective Stability) 12:26 — The Effect of an “Ontological Crisis” on AI 14:41 — Why Modern AI Will Not Be ‘Aligned By Default’ 18:49 — Debate: Have LLMs Solved the “Ontological Crisis” Problem? 25:56 — MIRI Alignment Greatest Hit: Timeless Decision Theory 35:17 — MIRI Alignment Greatest Hit: Corrigibility 37:53 — No Known Solution for Corrigible and Reflectively Stable Superintelligence39:58 — RecapShow NotesStay tuned for part 3 of my interview with Tsvi where we debate AGI timelines! Learn more about Tsvi’s organization, the Berkeley Genomics Project: https://berkeleygenomics.orgWatch part 1 of my interview with Tsvi: TranscriptEpisode HighlightsTsvi Benson-Tilsen 00:00:00If humans really f*cked up, when we try to reach into the AI and correct it, the AI does not want humans to modify the core aspects of what it values.Liron Shapira 00:00:09This concept is very deep, very important. It’s almost MIRI in a nutshell. I feel like MIRI’s whole research program is noticing: hey, when we run the AI, we’re probably going to get a bunch of generations of thrashing. But that’s probably only after we’re all dead and things didn’t happen the way we wanted. I feel like that is what MIRI is trying to tell the world. Meanwhile, the world is like, “la la la, LLMs, reinforcement learning—it’s all good, it’s working great. Alignment by default.”Tsvi 00:00:34Yeah, that’s certainly how I view it.Humanity Has Made 0% Progress on AI Alignment Liron Shapira 00:00:46All right. I want to move on to talk about your MIRI research. I have a lot of respect for MIRI. A lot of viewers of the show appreciate MIRI’s contributions. I think it has made real major contributions in my opinion—most are on the side of showing how hard the alignment problem is, which is a great contribution. I think it worked to show that. My question for you is: having been at MIRI for seven and a half years, how are we doing on theories of AI alignment?Tsvi Benson-Tilsen 00:01:10I can’t speak with 100% authority because I’m not necessarily up to date on everything and there are lots of researchers and lots of controversy. But from my perspective, we are basically at 0%—at zero percent done figuring it out. Which is somewhat grim. Basically, there’s a bunch of fundamental challenges, and we don’t know how to grapple with these challenges. Furthermore, it’s sort of sociologically difficult to even put our attention towards grappling with those challenges, because they’re weirder problems—more pre-paradigmatic. It’s harder to coordinate multiple people to work on the same thing productively.It’s also harder to get funding for super blue-sky research. And the problems themselves are just slippery.MIRI Alignment Greatest Hits: Reflective Probability Theory, Logical Uncertainty, Reflective Stability Liron 00:01:55Okay, well, you were there for seven years, so how did you try to get us past zero?Tsvi 00:02:00Well, I would sort of vaguely (or coarsely) break up my time working at MIRI into two chunks. The first chunk is research programs that were pre-existing when I started: reflective probability theory and reflective decision theory. Basically, we were trying to understand the mathematical foundations of a mind that is reflecting on itself—thinking about itself and potentially modifying itself, changing itself. We wanted to think about a mind doing that, and then try to get some sort of fulcrum for understanding anything that’s stable about this mind.Something we could say about what this mind is doing and how it makes decisions—like how it decides how to affect the world—and have our description of the mind be stable even as the mind is changing in potentially radical ways.Liron 00:02:46Great. Okay. Let me try to translate some of that for the viewers here. So, MIRI has been the premier organization studying intelligence dynamics, and Eliezer Yudkowsky—especially—people on social media like to dunk on him and say he has no qualifications, he’s not eve

I’m excited to share my recent AI doom interview with Eben Pagan, better known to many by his pen name David DeAngelo!For an entire generation of men, ‘David DeAngelo’ was the authority on dating—and his work transformed my approach to courtship back in the day. Now the roles reverse, as I teach Eben about a very different game, one where the survival of our entire species is at stake.In this interview, we cover the expert consensus on AI extinction, my dead-simple two-question framework for understanding the threat, why there’s no “off switch” for superintelligence, and why we desperately need international coordination before it’s too late.Timestamps0:00 - Episode Preview1:05 - How Liron Got Doom-Pilled2:55 - Why There’s a 50% Chance of Doom by 20504:52 - What AI CEOs Actually Believe8:14 - What “Doom” Actually Means10:02 - The Next Species is Coming12:41 - The Baby Dragon Fallacy14:41 - The 2-Question Framework for AI Extinction18:38 - AI Doesn’t Need to Hate You to Kill You21:05 - “Computronium”: The End Game29:51 - 3 Reasons There’s No Superintelligence “Off Switch”36:22 - Answering ‘What Is Intelligence?”43:24 - We Need Global CoordinationShow NotesEben has become a world-class business trainer and someone who follows the AI discourse closely. I highly recommend subscribing to his podcast for excellent interviews & actionable AI tips: @METAMIND_AI---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Former Machine Intelligence Research Institute (MIRI) researcher Tsvi Benson-Tilsen is championing an audacious path to prevent AI doom: engineering smarter humans to tackle AI alignment.I consider this one of the few genuinely viable alignment solutions, and Tsvi is at the forefront of the effort. After seven years at MIRI, he co-founded the Berkeley Genomics Project to advance the human germline engineering approach.In this episode, Tsvi lays out how to lower P(doom), arguing we must stop AGI development and stigmatize it like gain-of-function virus research. We cover his AGI timelines, the mechanics of genomic intelligence enhancement, and whether super-babies can arrive fast enough to save us.I’ll be releasing my full interview with Tsvi in 3 parts. Stay tuned for part 2 next week!Timestamps0:00 Episode Preview & Introducing Tsvi Benson-Tilsen1:56 What’s Your P(Doom)™4:18 Tsvi’s AGI Timeline Prediction6:16 What’s Missing from Current AI Systems10:05 The State of AI Alignment Research: 0% Progress11:29 The Case for PauseAI 15:16 Debate on Shaming AGI Developers25:37 Why Human Germline Engineering31:37 Enhancing Intelligence: Chromosome Vs. Sperm Vs. Egg Selection37:58 Pushing the Limits: Head Size, Height, Etc.40:05 What About Human Cloning?43:24 The End-to-End Plan for Germline Engineering45:45 Will Germline Engineering Be Fast Enough?48:28 Outro: How to Support Tsvi’s WorkShow NotesTsvi’s organization, the Berkeley Genomics Project — https://berkeleygenomics.orgIf you’re interested to connect with Tsvi about germline engineering, you can reach out to him at [email protected] Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Today I'm sharing my interview on Robert Wright's Nonzero Podcast where we unpack Eliezer Yudkowsky's AI doom arguments from his bestselling book, "If Anyone Builds It, Everyone Dies." Bob is an exceptionally thoughtful interviewer who asks sharp questions and pushes me to defend the Yudkowskian position, leading to a rich exploration of the AI doom perspective. I highly recommend getting a premium subscription to his podcast: 0:00 Episode Preview 2:43 Being a "Stochastic Parrot" for Eliezer Yudkowsky 5:38 Yudkowsky's Book: "If Anyone Builds It, Everyone Dies" 9:38 AI Has NEVER Been Aligned 12:46 Liron Explains "Intellidynamics" 15:05 Natural Selection Leads to Maladaptive Behaviors — AI Misalignment Foreshadowing 29:02 We Summon AI Without Knowing How to Tame It 32:03 The "First Try" Problem of AI Alignment 37:00 Headroom Above Human Capability 40:37 The PauseAI Movement: The Silent Majority 47:35 Going into Overtime Get full access to Doom Debates at lironshapira.substack.com/subscribe

Today I’m taking a rare break from AI doom to cover the dumbest kind of doom humanity has ever created for itself: climate change. We’re talking about a problem that costs less than $2 billion per year to solve. For context, that’s what the US spent on COVID relief every 7 hours during the pandemic. Bill Gates could literally solve this himself.My guest Andrew Song runs Make Sunsets, which launches weather balloons filled with sulfur dioxide (SO₂) into the stratosphere to reflect sunlight and cool the planet. It’s the same mechanism volcanoes use—Mount Pinatubo cooled Earth by 0.5°C for a year in 1991. The physics is solid, the cost is trivial, and the coordination problem is nonexistent.So why aren’t we doing it? Because people are squeamish about “playing God” with the atmosphere, even while we’re building superintelligent AI. Because environmentalists would rather scold you into turning off your lights than support a solution that actually works.This conversation changed how I think about climate change. I went from viewing it as this intractable coordination problem to realizing it’s basically already solved—we’re just LARPing that it’s hard! 🙈 If you care about orders of magnitude, this episode will blow your mind. And if you feel guilty about your carbon footprint: you can offset an entire year of typical American energy usage for about 15 cents. Yes, cents.Timestamps* 00:00:00 - Introducing Andrew Song, Cofounder of Make Sunsets* 00:03:08 - Why the company is called “Make Sunsets”* 00:06:16 - What’s Your P(Doom)™ From Climate Change* 00:10:24 - Explaining geoengineering and solar radiation management* 00:16:01 - The SO₂ dial we can turn* 00:22:00 - Where to get SO₂ (gas supply stores, sourcing from oil)* 00:28:44 - Cost calculation: Just $1-2 billion per year* 00:34:15 - “If everyone paid $3 per year”* 00:42:38 - Counterarguments: moral hazard, termination shock* 00:44:21 - Being an energy hog is totally fine* 00:52:16 - What motivated Andrew (his kids, Luke Iseman)* 00:59:09 - “The stupidest problem humanity has created”* 01:11:26 - Offsetting CO₂ from OpenAI’s Stargate* 01:13:38 - Playing God is goodShow NotesMake Sunsets* Website: https://makesunsets.com* Tax-deductible donations (US): https://givebutter.com/makesunsetsPeople Mentioned* Casey Handmer: https://caseyhandmer.wordpress.com/* Emmett Shear: https://twitter.com/eshear* Palmer Luckey: https://twitter.com/PalmerLuckeyResources Referenced* Book: Termination Shock by Neal Stephenson* Book: The Rational Optimist by Matt Ridley* Book: Enlightenment Now by Steven Pinker* Harvard SCoPEx project (the Bill Gates-funded project that got blocked)* Climeworks (direct air capture company): https://climeworks.comData/Monitoring* NOAA (National Oceanic and Atmospheric Administration): https://www.noaa.gov* ESA Sentinel-5P TROPOMI satellite data---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

I’ve been puzzled by David Deutsch’s AI claims for years. Today I finally had the chance to hash it out: Brett Hall, one of the foremost educators of David Deutsch’s ideas around epistemology & science, was brave enough to debate me!Brett has been immersed in Deutsch’s philosophy since 1997 and teaches it on his Theory of Knowledge podcast, which has been praised by tech luminary Naval Ravikant. He agrees with Deutsch on 99.99% of issues, especially the dismissal of AI as an existential threat.In this debate, I stress-test the Deutschian worldview, and along the way we unpack our diverging views on epistemology, the orthogonality thesis, and pessimism vs optimism.Timestamps0:00 — Debate preview & introducing Brett Hall4:24 — Brett’s opening statement on techno-optimism13:44 — What’s Your P(Doom)?™15:43 — We debate the merits of Bayesian probabilities20:13 — Would Brett sign the AI risk statement?24:44 — Liron declares his “damn good reason” for AI oversight35:54 — Debate milestone: We identify our crux of disagreement!37:29 — Prediction vs prophecy44:28 — The David Deutsch CAPTCHA challenge1:00:41 — What makes humans special?1:15:16 — Reacting to David Deutsch’s recent statements on AGI1:24:04 — Debating what makes humans special1:40:25 — Brett reacts to Roger Penrose’s AI claims1:48:13 — Debating the orthogonality thesis1:56:34 — The powerful AI data center hypothetical2:03:10 — “It is a dumb tool, easily thwarted”2:12:18 — Clash of worldviews: goal-driven vs problem-solving2:25:05 — Ideological Turing test: We summarize each other’s positions2:30:44 — Are doomers just pessimists?Show NotesBrett’s website — https://www.bretthall.orgBrett’s Twitter — https://x.com/TokTeacherThe Deutsch Files by Brett Hall and Naval Ravikant* https://nav.al/deutsch-files-i* https://nav.al/deutsch-files-ii* https://nav.al/deutsch-files-iii* https://nav.al/deutsch-files-ivBooks:* The Fabric of Reality by David Deutsch* The Beginning of Infinity by David Deutsch* Superintelligence by Nick Bostrom* If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

We took Eliezer Yudkowsky and Nate Soares’s new book, If Anyone Builds It, Everyone Dies, on the streets to see what regular people think.Do people think that artificial intelligence is a serious existential risk? Are they open to considering the argument before it’s too late? Are they hostile to the idea? Are they totally uninterested?Watch this episode to see the full spectrum of reactions from a representative slice of America!---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Welcome to the Doom Debates + Wes Roth + Dylan Curious crossover episode!Wes & Dylan’s host a popular YouTube AI news show that’s better than mainstream media. They interview thought leaders like Prof. Nick Bostrom, Dr. Mike Israetel — and now, yours truly!This episode is Part 2, where Wes & Dylan come on Doom Debates to break down the latest AI news.In Part 1, I went on Wes & Dylan’s channel to talk about my AI doom worries:The only reasonable move is to subscribe to both channels and watch both parts!Timestamps00:00 — Cold open00:45 — Introducing Wes & Dylan & hearing their AI YouTuber origin stories05:38 — What’s Your P(Doom)? ™10:30 — Living with high P(doom)12:10 — AI News Roundup: If Anyone Builds It, Everyone Dies Reactions17:02 — AI Redlines at the UN & risk of AGI authoritarianism26:56 — Robot ‘violence test’29:20 — Anthropic gets honest about job loss impact32:43 — AGI hunger strikes, the rationale for Anthropic protests41:24 — Liron explains his proposal for a safer future, treaty & enforcement debate49:23 — US Government officials ignore scientists and use “catastrophists” pejorative55:59 — Experts’ p(doom) predictions59:41 — Wes gives his take on AI safety warnings1:02:04 — Wrap-up, Subscribe to Wes Roth and Dylan CuriousShow NotesWes & Dylans’s channel — https://www.youtube.com/wesrothIf Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares — https://ifanyonebuildsit.com---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

I’m excited to announce the launch of Doom Hut, the official Doom Debates merch store!This isn’t just another merch drop. It’s a way to spread an urgent message that humanity faces an imminent risk from superintelligent AI, and we need to build common knowledge about it now.What We’re OfferingThe Doom Hut is the premiere source of “high P(doom) fashion.” We’ve got t-shirts in men’s and women’s styles, tall tees, tote bags, and hats that keep both the sun and the doom out of your eyes.Our signature items are P(doom) pins. Whether your probability of doom is less than 10%, greater than 25%, or anywhere in between, you can represent your assessment with pride.Our shirts feature “Doom Debates” on the front and “What’s your P(doom)?” on the back. It’s a conversation starter that invites questions rather than putting people on the defensive.Why This MattersThis is crunch time. There’s no more time to beat around the bush or pretend everything’s okay.When you wear this merch, you’re not just making a fashion statement. You’re signaling that P(doom) is uncomfortably high and that we need to stop building dangerous AI before we know how to control it.People will wonder what “Doom Debates” means. They’ll see that you’re bold enough to acknowledge the world faces real threats. Maybe they’ll follow your example. Maybe they’ll start asking questions and learning more.Supporting the MissionThe merch store isn’t really about making money for us (we make about $2 per item sold). The show’s production and marketing costs are funded by donations from viewers like you. Visit doomdebates.com/donate to learn more.Donations to Doom Debates are fully tax-deductible through our fiscal sponsor, Manifund.org. You can also support us by becoming a premium subscriber to DoomDebates.com, which helps boost our Substack rankings and visibility.Join the CommunityJoin our Discord community here! It’s a vibrant space where people debate, share memes, discuss new episodes, and talk about where they “get off the doom train.”Moving Forward TogetherThe outpouring of support since we started accepting donations has been incredible. You get what we’re trying to do here—raising awareness and helping the average person understand that AI risk is an imminent threat to them and their loved ones.This is real. This is urgent. And together, we’re moving the Overton window. Thanks for your support. Get full access to Doom Debates at lironshapira.substack.com/subscribe

Eliezer Yudkowsky can warn humankind that If Anyone Builds It, Everyone Dies and get on the New York Times bestseller list, but he won’t get upvoted to the top of LessWrong.According to the leaders of LessWrong, that’s intentional. The rationalist community thinks aggregating community support for important claims is “political fighting”.Unfortunately, the idea that some other community will strongly rally behind Eliezer Yudkowsky’s message while LessWrong “stays out of the fray” and purposely prevents mutual knowledge of support from being displayed, is unrealistic.Our refusal to aggregate the rationalist community beliefs into signals and actions is why we live in a world where rationalists with double-digit P(Doom)s join AI-race companies instead of AI-pause movements.We let our community become a circular firing squad. What did we expect?Timestamps00:00:00 — Cold Open00:00:32 — Introducing Holly Elmore, Exec. Director of PauseAI US00:03:12 — “If Anyone Builds It, Everyone Dies”00:10:07 — What’s Your P(Doom)™00:12:55 — Liron’s Review of IABIED00:15:29 — Encouraging Early Joiners to a Movement00:26:30 — MIRI’s Communication Issues00:33:52 — Government Officials’ Reviews of IABIED00:40:33 — Emmett Shear’s Review of IABIED00:42:47 — Michael Nielsen’s Review of IABIED00:45:35 — New York Times Review of IABIED00:49:56 — Will MacAskill’s Review of IABIED01:11:49 — Clara Collier’s Review of IABIED01:22:17 — Vox Article Review01:28:08 — The Circular Firing Squad01:37:02 — Why Our Kind Can’t Cooperate01:49:56 — LessWrong’s Lukewarm Show of Support02:02:06 — The “Missing Mood” of Support02:16:13 — Liron’s “Statement of Support for IABIED”02:18:49 — LessWrong Community’s Reactions to the Statement02:29:47 — Liron & Holly’s Hopes for the Community02:39:01 — Call to ActionSHOW NOTESPauseAI US — https://pauseai-us.orgPauseAI US Upcoming Events — https://pauseai-us.org/eventsInternational PauseAI — https://pauseai.infoHolly’s Twitter — https://x.com/ilex_ulmusReferenced Essays & Posts* Liron’s Eliezer Yudkowsky interview post on LessWrong — https://www.lesswrong.com/posts/kiNbFKcKoNQKdgTp8/interview-with-eliezer-yudkowsky-on-rationality-and* Liron’s “Statement of Support for If Anyone Builds It, Everyone Dies” — https://www.lesswrong.com/posts/aPi4HYA9ZtHKo6h8N/statement-of-support-for-if-anyone-builds-it-everyone-dies* “Why Our Kind Can’t Cooperate” by Eliezer Yudkowsky (2009) — https://www.lesswrong.com/posts/7FzD7pNm9X68Gp5ZC/why-our-kind-can-t-cooperate* “Something to Protect” by Eliezer Yudkowsky — https://www.lesswrong.com/posts/SGR4GxFK7KmW7ckCB/something-to-protect* Center for AI Safety Statement on AI Risk — https://safe.ai/work/statement-on-ai-riskOTHER RESOURCES MENTIONED* Stephen Pinker’s new book on mutual knowledge, When Everyone Knows That Everyone Knows... — https://stevenpinker.com/publications/when-everyone-knows-everyone-knows-common-knowledge-and-mysteries-money-power-and* Scott Alexander’s “Ethnic Tension and Meaningless Arguments” — https://slatestarcodex.com/2014/11/04/ethnic-tension-and-meaningless-arguments/PREVIOUS EPISODES REFERENCEDHolly’s previous Doom Debates appearance debating the California SB 1047 bill — https://www.youtube.com/watch?v=xUP3GywD0yMLiron’s interview with Eliezer Yudkowsky about the IABI launch — https://www.youtube.com/watch?v=wQtpSQmMNP0---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Ex-Twitch Founder and OpenAI Interim CEO Emmett Shear is one of the rare established tech leaders to lend his name and credibility to Eliezer Yudkowsky’s warnings about AI existential risk.Even though he disagrees on some points, he chose to endorse the new book If Anyone Builds It, Everyone Dies:“Soares and Yudkowsky lay out, in plain and easy-to-follow terms, why our current path toward ever-more-powerful AIs is extremely dangerous.”In this interview from the IABED launch party, we dive into Emmett’s endorsement, why the current path is so dangerous, and what he hopes to achieve by taking a different approach at his new startup, Softmax.Watch the full IABED livestream here: https://www.youtube.com/watch?v=lRITRf-jH1gWatch my reaction to Emmett’s talk about Softmax to see why I’m not convinced his preferred alignment track is likely to work: https://www.youtube.com/watch?v=CBN1E1fvh2g Get full access to Doom Debates at lironshapira.substack.com/subscribe

ATTENTION DOOM DEBATES LISTENERS:We interrupt our usual programming to request that you consider supporting the show.Do you want your family, children & friends to continue living life through these next 10-20 years and beyond? Want to experience the good future where we benefit from research advances that make us healthier, happier, wiser, and longer-lived? Want our descendants to continue flourishing for trillions of millennia?I sure as hell do. I’m thrilled and inspired by the grand future within our reach… Sadly, I’m VERY worried that artificial intelligence may soon cause human extinction, leaving the future permanently devoid of the incredible potential value it once had.I want to safeguard humanity’s chance at participating in the expansive future. I want to swerve away from a premature “GAME OVER” ending. That’s why I started Doom Debates.The MissionDoom Debates’s mission is twofold:* Raise awareness of AI existential risk* Raise the quality of discourse that people can engage with and trustThe mission is achieved when the average person realizes AI is life-threatening.Until the average person on Earth sees unaligned, uncontrollable superintelligent AI as life-threatening — that is, an imminent threat to their life and to everything they hold dear — it won’t be feasible for leaders to drive major decisive action to protect us from extinction from superintelligent AI. That’s why raising mainstream awareness of AI x-risk is Doom Debates’s point of highest leverage.Encouragingly, surveys already show that mainstream opinion is already on our side about the level and severity of AI x-risk. What’s missing is urgency. Moving the average person’s opinion from “worried, but in an abstract long-term way” to “alarmed, in an urgent way” creates grassroots demand for drastically better AI policy, and drastically better-informed policymakers.We needed this shift to happen yesterday; it feels late to have to drive it now. But we have no choice.By providing high-quality interviews and debates with mainstream appeal, we’ll make millions of people realize as soon as possible: “Hey, P(doom) is too damn high RIGHT NOW. And we need to do something about it URGENTLY!”One Weird Trick You Can Do To HelpIf you care about the show’s mission, the #1 way you can help right now is by donating some of your hard-earned cash money 🙏.Spending on production & marketing accelerates the show’s growth, which achieves our mission faster (hopefully before it’s too late). Read on for more details.Where Does The Money Go?I don’t make money from Doom Debates. My “real job” is running an online relationship coaching service, and my passion project is making this show. Any income from Doom Debates, i.e. viewer donations and ad revenue, is fully reinvested into production and marketing.* Producer OriThanks to a generous viewer donation, I’ve hired a full-time producer: Ori Nagel.Ori is a rockstar who I’ve known for many years, and he collaborated with me behind the scenes throughout the show’s first year (he’s also in the first episode) before officially joining Doom Debates as our Producer. I hired him away from the media & outreach team at ControlAI, where he 10x’d their growth on social media channels.You may have noticed the show has already started getting better at editing & thumbnails, putting out more episodes and clips, and landing more prominent guests. We’re getting more done across the board because 1 hour of Ori’s time working on everything except hosting the show = same output as 1 hour of my time.But it’s not all smooth sailing — now that Doom Debates is Ori’s full-time job, he apparently needs to “get paid every month” 🤷‍♂️, so I’m appealing to you to help Ori stay on the job. Help us keep delivering faster progress toward the mission.* Paid MarketingThe key to the show’s growth isn’t marketing; it’s the content. To date, we’ve had robust organic growth with minimal marketing spend. Audience size has been doubling every 3 months, and we’re seeing a snowball effect where bigger audiences attract higher-quality guests and vice versa. We’re also happy to see that the show has unusually high engagement and long-term viewer retention.That said, investing in marketing lets us pull forward the same viewership we’d eventually get from organic growth. We’ll soon invest in YouTube ads to go beyond the pace of organic growth. We also keep spending opportunistically on marketing initiatives like sponsoring the Manifest conference and giving away T-shirts.Donation TiersSUBSTACK SUPPORTERSYou can donate as little as $10/month by subscribing to the DoomDebates.com Substack, which shows your support and raises our profile on their leaderboard. I’ll send you a free Doom Debates T-shirt and P(Doom) pin so you can represent the show. Early supporters say the T-shirt is great at starting AI x-risk conversations in a non-confrontational way. 😊MISSION PARTNERSIf you’re serious about lowering P(Doom) and you have the money to spare, $1k+ is the l

Science communicator and poker champion Liv Boeree has been concerned about the existential risk of AI for nearly a decade.In this conversation from my If Anyone Builds It, Everyone Dies launch party, Liv explains why even a 5% chance of AI extinction is unacceptable, why the industry often buries its head in the sand, and how advocates can actually make an impact.---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Eliezer Yudkowsky and Nate Soares just launched their world-changing book, If Anyone Builds It, Everyone Dies. PLEASE BUY YOUR COPY NOW!!!We had an unofficial launch party here on Doom Debates to celebrate the occasion with an incredible group of guests (see timestamps below). Highly recommend watching if you missed it!Timestamps* 00:00 — Cold Open* 00:36 — Max Tegmark, MIT Physicist* 24:58 — Roman Yampolskiy, Cybersecurity Professor * 42:20 Michael of @lethal-intelligence* 48:30 — Liv Boeree, Host of the Win-Win Podcast ( @LivBoeree )* 1:10:44 — Michael Trazzi, Filmmaker who just went on a hunger strike against Google DeepMind (@TheInsideView)* 1:27:21 — Producer Ori* 1:43:53 — Emmett Shear, Founder of Twitch & Softmax * 1:55:32 — Holly Elmore, Executive Director of PauseAI US* 2:13:44 — Gary Marcus, Professor Emeritus of Psychology & Neural Science at NYU* 2:36:13 — Robert Wright, Author of the NonZero Newsletter (@Nonzero)* 2:55:50 — Roko Mijic, the Man Behind “Roko’s Basilisk” * 3:19:17 — Rob Miles, AI Safety Educator (@RobertMilesAI)* 3:48:31 — Doom Debates Robot Closes Out the Stream!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Max Tegmark is an MIT physics professor, science communicator, best-selling author, and the President and co-founder of the Future of Life Institute. There are few individuals who have done more to get the world leaders to come to a shared sense of reality about the extinction threat of AI.His endorsement of Eliezer’s book, If Anyone Builds It, Everyone Dies, states: "The most important book of the decade."Max shared his sharp criticisms of AI companies yesterday on my unofficial IABI launch party livestream!---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

It is Monday, September 15th and my interview with Eliezer Yudkowsky has dropped!!!👉 WATCH NOW 👈We had a very unique 2.5-hour discussion covering rationality topics, the AI doom argument, what the AI companies think they're doing, why they don't seem to understand the AI doom argument, what society is doing, how we can help. And of course, his book, "If Anyone Builds It, Everyone Dies," which is officially launching tomorrow.For now, go ahead and watch the Eliezer Yudkowsky interview, and please smash that like button. We gotta get this a lot of likes.Thanks very much for being a Doom Debates watcher. We've also got a launch party coming up. I'll get back to you with more details on that soon. Get full access to Doom Debates at lironshapira.substack.com/subscribe

The Eliezer Yudkowsky interview premiere is tomorrow (Mon Sep 15) at 9am PT!!!👉 https://www.youtube.com/watch?v=wQtpSQmMNP0 👈I can't believe it. We are entering the launch week for If Anyone Builds It, Everyone Dies - a new book by Eliezer Yudkowsky and Nate Soares. This is a hell of a book. I highly recommend it.Everybody should pick it up. Honestly, if you haven't gone and bought that book by now, not gonna lie, I'm kind of disappointed. Should you really even be watching this channel? Are you not getting the message that it's critical for you to buy this book?It's going to be on the New York Times bestseller list, and the only question is, what position will it be? That's going to depend on how many people like you take action. And by action, I mean, you know, pay $14.99, get it on Kindle. Really do your part. It's not that much.Once you've done that, remember, tomorrow something very special is happening on my personal channel - the long awaited interview between me and Eliezer Yudkowsky! We've all been waiting so long for me to talk to Eliezer. It finally happened as part of his book tour.Technically, it's not part of Doom Debates, it's just on my own channel because it's not branded as a Doom thing. It's just a Yudkowsky interview. So you're not gonna see me ask P(Doom). That's not gonna be part of it. We're just going to talk in terms of existential risk and looking at dangers. That's the preferred terminology here. There's a difference of opinion between him and me, and we gotta respect that.So action item for you. Once you've bought that book, "If Anyone Builds It, Everyone Dies," head over to the link in the show notes for this post or in the description for this post. There's going to be a link to a YouTube premiere. That YouTube premiere is the Eliezer Yudkowsky interview, which is happening 9:00 AM tomorrow, Monday, September 15th.That's really not that long after you're listening to this post. We're talking hours away. You want to be subscribed to that premiere. You don't want to miss it because it's not going to be a post in the Doom Debates feed. It's going to be its own YouTube premiere on my personal channel.So once again, check out those show notes. Check out the description to what you're watching right now. Click on that link and bookmark the premiere episode because I'm going to be there live in the live chat watching my own episode. Just like you're watching the episode, my producer Ori is going to be there live watching the episode. A bunch of your fellow Doom Debates fans are going to be in the live chat watching the episode.This is a really special event, both for Eliezer Yudkowsky and MIRI, and for Doom Debates and for the human population. At least the human population can say that there was a New York Times bestseller calling out what's about to happen to all of us if we don't stop it. I think it's a very special week.I hope you'll help it be as high profile as possible so that the algorithms will take notice and society will take notice and a grassroots movement will take notice and our politicians and other leaders will take notice.All right. Signing off for now. I'll see you all in the YouTube premiere of the Eliezer Yudkowsky interview. Thanks for watching!---P.S. This post also has more details about Tuesday’s Doom Debates launch party! Get full access to Doom Debates at lironshapira.substack.com/subscribe

In this special cross-post from Jona Ragogna’s channel, I'm interviewed about why superintelligent AI poses an imminent extinction threat, and how AI takeover is going to unfold.Newly exposed to AI x-risk, Jona asks sharp questions about why we’re racing toward superintelligence despite the danger, and what ordinary people can do now to lower p(doom). This is one of the most crisp explainers of the AI-doom argument I’ve done to date.Timestamps0:00 Intro0:41 Why AI is likely to cause human extinction2:55 How AI takeover happens4:55 AI systems have goals6:33 Liron explains p(Doom)8:50 The worst case scenario is AI sweeps us away12:46 The best case scenario is hard to define14:24 How to avoid doom15:09 Frontier AI companies are just doing "ad hoc" alignment20:30 Why "warning shots" from AI aren't scary yet23:19 Should young adults work on AI alignment research?24:46 We need a grassroots movement28:31 Life choices when AI doom is imminent32:35 Are AI forecasters just biased?34:12 The Doom Train™ and addressing counterarguments40:28 Anthropic's new AI welfare announcement isn't a major breakthrough44:35 It's unknown what's going on inside LLMs and AI systems53:22 Effective Altruism's ties to AI risk56:58 Will AI be a "worthy descendant"?1:01:08 How to calculate P(Doom)1:02:49 Join the unofficial If Anyone Builds It, Everyone Dies book launch party!Show NotesSubscribe to Jona Ragogna — https://youtube.com/@jonaragognaIF ANYONE BUILDS IT LAUNCH WEEK EVENTS:Mon Sep 15 @ 9am PT / 12pm ET / 1600 UTCMy Eliezer Yudkowsky premieres on YouTube! Stay tuned for details.Tue Sep 16 @ 2pm PT / 5pm ET / 2100 UTCThe Doom Debates unofficial IABI Launch Party!!!More details about launch week HERE!---Doom Debates’s Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Ladies and gentlemen, we are days away from the long-awaited release of Eliezer Yudkowsky and Nate Soares's new book, “If Anyone Builds It, Everyone Dies”!!!Mon Sep 15 @9am PT: My Interview with Eliezer YudkowskyWe'll be kicking things off the morning of Monday, September 15th with a live watch party of my very special new interview with the one and only Eliezer Yudkowsky!All of us will be in the YouTube live chat. I'll be there, producer Ori will be there, and you'll get a first look at this new & exciting interview: Questions he's never been asked before.We'll talk about my history first meeting Eliezer in 2008, how things have evolved, what's going on now, why everybody has their head in the sand. And we'll even go to the LessWrong sequences and do some rationality deep cuts.This will be posted on my personal YouTube channel. But if you're subscribed to Doom Debates, you'll get the link in your feed as we get closer to Monday.Mark your calendars for Monday, September 15th, 9:00am Pacific, 12:00 PM Eastern. That's 1600 UTC for my Europals out there, midnight for my China peeps.I think YouTube commenter @pairy5420 said it best: "Bro, I could have three essays due, my crush just text me, the FBI outside my house, me having a winning lottery ticket to turn in all at the same time, and I still wouldn't miss the interview."That's the spirit, @pairy5420. You won't want to miss the Eliezer interview YouTube premiere. I'll see you all there. It's gonna be a great start to the book's launch week.Tue Sep 16 @2pm PT: The Doom Debates Unofficial Book Launch PartyNow let's talk about the launch day: Tuesday, September 16th. Once you've picked up your copy of the book, get ready for the Doom Debates unofficial "If Anyone Builds It, Everyone Dies" launch party!!!This is going to be a very special, unprecedented event here on this channel. It's gonna be a three hour live stream hosted by me and producer Ori, our new full-time producer. He's gonna be the co-host of the party.He’s finally coming out from behind the scenes, making his debut on the channel. Unless you count the first episode of the show - he was there too. The two of us are going to be joined by a who's who of very special guests. Get a load of these guests who are gonna be dropping by the unofficial launch party:* John Sherman from the AIX Risk Network and Michael, my other co-host of Warning Shots* Roman Yampolskiy, a top researcher and communicator in the field of AI x-risk* Emmett Shear, founder and CEO of a new AI alignment company called Softmax. He was previously the CEO of Twitch, and the interim CEO of OpenAI* Roon, member of the technical staff at OpenAI and friend of the show* Gary Marcus, cognitive scientist, author, entrepreneur, and famous AGI skeptic* Liv Boeree, the YouTube star, poker champion, and science communicator* Robert Wright, bestselling author, podcaster, wide-ranging intellectual, about announce his new book about AI* Holly Elmore, executive director of PauseAI US* Roko Mijic, you guys know Roko 😉And that's not all. There's going to be even more that I can't tell you about now, but it will not disappoint.So I really hope to see you all there at the unofficial "If Anyone Builds It, Everyone Dies" launch party, Tuesday, September 16th.Same as the book’s launch day: Tuesday, September 16th at 2:00pm Pacific, 5:00pm Eastern.Pick up your copy of the book that morning. Don't come to the party without it. We're gonna have a bouncer stationed at the door, and if you don't show him that you've got a copy of "If Anyone Builds It, Everyone Dies," he's gonna give you a big thumbs down.BUY THE BOOK!!!In all seriousness though, please support the book if you like Doom Debates. If you feel like you've gotten some value out of the show and you wanna give back a little bit, that is my ask. Head over to ifanyonebuildsit.com and buy the book from their links there. Go to Amazon, Barnes and Noble, wherever you normally buy books, just buy the damn thing. It's $14.99 on Kindle. It's not gonna break the bank.Then spread the word. Tell your friends and family, tell your coworkers at the office. Try to get a few more copies sold. We don't have another book launch coming, guys. This is it.This is our chance to take a little bit of action when it can actually move the needle and help. If you've been procrastinating this whole time, you gotta stop. You gotta go buy it now because the New York Times is gonna be checking this week.This is the last week of pre-orders. You really want to give it that launch bump. Don't try to drag it out after launch week. Time is of the essence.The Doom Debates MissionUltimately that's why I do this show. This isn't just entertainment for smart people. There is actually an important mission. We're trying to optimize the mission here. Help me out.Or at the very least, help high quality discourse because a lot of people across the spectrum agree, this is a high quality book contributing to the discourse, and we need more books

Louis Berman is a polymath who brings unique credibility to AI doom discussions. He's been coding AI for 25 years, served as CTO of major tech companies, recorded the first visual sighting of what became the dwarf planet Eris, and has now pivoted to full-time AI risk activism. He's lobbied over 60 politicians across multiple countries for PauseAI and authored two books on existential risk.Louis and I are both baffled by the calm, measured tone that dominates AI safety discourse. As Louis puts it: "No one is dealing with this with emotions. No one is dealing with this as, oh my God, if they're right. Isn't that the scariest thing you've ever heard about?"Louis isn't just talking – he's acting on his beliefs. He just bought a "bug out house" in rural Maryland, though he's refreshingly honest that this isn't about long-term survival. He expects AI doom to unfold over months or years rather than Eliezer's instant scenario, and he's trying to buy his family weeks of additional time while avoiding starvation during societal collapse.He's spent extensive time in congressional offices and has concrete advice about lobbying techniques. His key insight: politicians' staffers consistently claim "if just five people called about AGI, it would move the needle". We need more people like Louis!Timestamps* 00:00:00 - Cold Open: The Missing Emotional Response* 00:00:31 - Introducing Louis Berman: Polymath Background and Donor Disclosure* 00:03:40 - The Anodyne Reaction: Why No One Seems Scared* 00:07:37 - P-Doom Calibration: Gary Marcus and the 1% Problem* 00:11:57 - The Bug Out House: Prepping for Slow Doom* 00:13:44 - Being Amazed by LLMs While Fearing ASI* 00:18:41 - What’s Your P(Doom)™* 00:25:42 - Bayesian Reasoning vs. Heart of Hearts Beliefs* 00:32:10 - Non-Doom Scenarios and International Coordination* 00:40:00 - The Missing Mood: Where's the Emotional Response?* 00:44:17 - Prepping Philosophy: Buying Weeks, Not Years* 00:52:35 - Doom Scenarios: Slow Takeover vs. Instant Death* 01:00:43 - Practical Activism: Lobbying Politicians and Concrete Actions* 01:16:44 - Where to Find Louis's Books and Final Wrap-up* 01:18:17 - Outro: Super Fans and Mission PartnersLinksLouis’s website — https://xriskbooks.com — Buy his books!ControlAI’s form to easily contact your representative and make a difference — https://controlai.com/take-action/usa — Highly recommended!Louis’s interview about activism with John Sherman and Felix De Simone — https://www.youtube.com/watch?v=Djd2n4cufTMIf Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares — https://ifanyonebuildsit.comBecome a Mission Partner!Want to meaningfully help the show’s mission (raise awareness of AI x-risk & raise the level of debate around this crucial subject) become reality? Donate $1,000+ to the show (no upper limit) and I’ll invite you to the private Discord channel. Email me at [email protected] if you have questions or want to donate crypto.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Rob Miles is the most popular AI safety educator on YouTube, with millions of views across his videos explaining AI alignment to general audiences. He dropped out of his PhD in 2011 to focus entirely on AI safety communication – a prescient career pivot that positioned him as one of the field's most trusted voices over a decade before ChatGPT made AI risk mainstream.Rob sits firmly in the 10-90% P(Doom) range, though he admits his uncertainty is "hugely variable" and depends heavily on how humanity responds to the challenge. What makes Rob particularly compelling is the contrast between his characteristic British calm and his deeply serious assessment of our situation. He's the type of person who can explain existential risk with the measured tone of a nature documentarian while internally believing we're probably headed toward catastrophe.Rob has identified several underappreciated problems, particularly around alignment stability under self-modification. He argues that even if we align current AI systems, there's no guarantee their successors will inherit those values – a discontinuity problem that most safety work ignores. He's also highlighted the "missing mood" in AI discourse, where people discuss potential human extinction with the emotional register of an academic conference rather than an emergency.We explore Rob's mainline doom scenario involving recursive self-improvement, why he thinks there's enormous headroom above human intelligence, and his views on everything from warning shots to the Malthusian dynamics that might govern a post-AGI world. Rob makes a fascinating case that we may be the "least intelligent species capable of technological civilization" – which has profound implications for what smarter systems might achieve.Our key disagreement centers on strategy: Rob thinks some safety-minded people should work inside AI companies to influence them from within, while I argue this enables "tractability washing" that makes the companies look responsible while they race toward potentially catastrophic capabilities. Rob sees it as necessary harm reduction; I see it as providing legitimacy to fundamentally reckless enterprises.The conversation also tackles a meta-question about communication strategy. Rob acknowledges that his measured, analytical approach might be missing something crucial – that perhaps someone needs to be "running around screaming" to convey the appropriate emotional urgency. It's a revealing moment from someone who's spent over a decade trying to wake people up to humanity's most important challenge, only to watch the world continue treating it as an interesting intellectual puzzle rather than an existential emergency.Timestamps* 00:00:00 - Cold Open* 00:00:28 - Introducing Rob Miles* 00:01:42 - Rob's Background and Childhood* 00:02:05 - Being Aspie* 00:04:50 - Less Wrong Community and "Normies"* 00:06:24 - Chesterton's Fence and Cassava Root* 00:09:30 - Transition to AI Safety Research* 00:11:52 - Discovering Communication Skills* 00:15:36 - YouTube Success and Channel Growth* 00:16:46 - Current Focus: Technical vs Political* 00:18:50 - Nuclear Near-Misses and Y2K* 00:21:55 - What’s Your P(Doom)™* 00:27:31 - Uncertainty About Human Response* 00:31:04 - Views on Yudkowsky and AI Risk Arguments* 00:42:07 - Mainline Catastrophe Scenario* 00:47:32 - Headroom Above Human Intelligence* 00:54:58 - Detailed Doom Scenario* 01:01:07 - Self-Modification and Alignment Stability* 01:17:26 - Warning Shots Problem* 01:20:28 - Moving the Overton Window* 01:25:59 - Protests and Political Action* 01:33:02 - The Missing Mood Problem* 01:40:28 - Raising Society's Temperature* 01:44:25 - "If Anyone Builds It, Everyone Dies"* 01:51:05 - Technical Alignment Work* 01:52:00 - Working Inside AI Companies* 01:57:38 - Tractability Washing at AI Companies* 02:05:44 - Closing Thoughts* 02:08:21 - How to Support Doom Debates: Become a Mission PartnerLinksRob’s YouTube channel — https://www.youtube.com/@RobertMilesAIRob’s Twitter — https://x.com/robertskmilesRational Animations (another great YouTube channel, narrated by Rob) — https://www.youtube.com/RationalAnimationsBecome a Mission Partner!Want to meaningfully help the show’s mission (raise awareness of AI x-risk & raise the level of debate around this crucial subject) become reality? Donate $1,000+ to the show (no upper limit) and I’ll invite you to the private Discord channel. Email me at [email protected] if you have questions or want to donate crypto. Get full access to Doom Debates at lironshapira.substack.com/subscribe

Vitalik Buterin is the founder of Ethereum, the world's second-largest cryptocurrency by market cap, currently valued at around $500 billion. But beyond revolutionizing blockchain technology, Vitalik has become one of the most thoughtful voices on AI safety and existential risk.He's donated over $665 million to pandemic prevention and other causes, and has a 12% P(Doom) – putting him squarely in what I consider the "sane zone" for AI risk assessment. What makes Vitalik particularly interesting is that he's both a hardcore techno-optimist who built one of the most successful decentralized systems ever created, and someone willing to seriously consider AI regulation and coordination mechanisms.Vitalik coined the term "d/acc" – defensive, decentralized, democratic, differential acceleration – as a middle path between uncritical AI acceleration and total pause scenarios. He argues we need to make the world more like Switzerland (defensible, decentralized) and less like the Eurasian steppes (vulnerable to conquest).We dive deep into the tractability of AI alignment, whether current approaches like DAC can actually work when superintelligence arrives, and why he thinks a pluralistic world of competing AIs might be safer than a single aligned superintelligence. We also explore his vision for human-AI merger through brain-computer interfaces and uploading.The crux of our disagreement is that I think we're heading for a "plants vs. animals" scenario where AI will simply operate on timescales we can't match, while Vitalik believes we can maintain agency through the right combination of defensive technologies and institutional design.Finally, we tackle the discourse itself – I ask Vitalik to debunk the common ad hominem attacks against AI doomers, from "it's just a fringe position" to "no real builders believe in doom." His responses carry weight given his credibility as both a successful entrepreneur and someone who's maintained intellectual honesty throughout his career.Timestamps* 00:00:00 - Cold Open* 00:00:37 - Introducing Vitalik Buterin* 00:02:14 - Vitalik's altruism* 00:04:36 - Rationalist community influence* 00:06:30 - Opinion of Eliezer Yudkowsky and MIRI* 00:09:00 - What’s Your P(Doom)™* 00:24:42 - AI timelines* 00:31:33 - AI consciousness* 00:35:01 - Headroom above human intelligence* 00:48:56 - Techno optimism discussion* 00:58:38 - e/acc: Vibes-based ideology without deep arguments* 01:02:49 - d/acc: Defensive, decentralized, democratic acceleration* 01:11:37 - How plausible is d/acc?* 01:20:53 - Why libertarian acceleration can paradoxically break decentralization* 01:25:49 - Can we merge with AIs?* 01:35:10 - Military AI concerns: How war accelerates dangerous development* 01:42:26 - The intractability question* 01:51:10 - Anthropic and tractability-washing the AI alignment problem* 02:00:05 - The state of AI x-risk discourse* 02:05:14 - Debunking ad hominem attacks against doomers* 02:23:41 - Liron’s outroLinksVitalik’s website: https://vitalik.eth.limoVitalik’s Twitter: https://x.com/vitalikbuterinEliezer Yudkowsky’s explanation of p-Zombies: https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies—Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Today I’m sharing my interview on Robert Wright’s Nonzero Podcast from last May. Rob is an especially sharp interviewer who doesn't just nod along, he had great probing questions for me.This interview happened right after Ilya Sutskever and Jan Leike resigned from OpenAI in May 2024, continuing a pattern that goes back to Dario Amodei leaving to start Anthropic. These aren't fringe doomers; these are the people hired specifically to solve the safety problem, and they keep concluding it's not solvable at the current pace.00:00:00 - Liron’s preface00:02:10 - Robert Wright introduces Liron00:04:02 - PauseAI protests at OpenAI headquarters00:05:15 - OpenAI resignations (Ilya Sutskever, Jan Leike, Dario Amodei, Paul Christiano, Daniel Kokotajlo)00:15:30 - P vs NP problem as analogy for AI alignment difficulty00:22:31 - AI pause movement and protest turnout00:29:02 - Defining AI doom and sci-fi scenarios00:32:05 - What’s My P(Doom)™00:35:18 - Fast vs slow AI takeoff and Sam Altman's position00:42:33 - Paperclip thought experiment and instrumental convergence explanation00:54:40 - Concrete examples of AI power-seeking behavior (business assistant scenario)01:00:58 - GPT-4 TaskRabbit deception example and AI reasoning capabilities01:09:00 - AI alignment challenges and human values discussion01:17:33 - Wrap-up and transition to premium subscriber contentShow NotesThis episode on Rob’s Nonzero Newsletter. You can subscribe for premium access to the last 1 hour of our discussion! — https://www.nonzero.org/p/in-defense-of-ai-doomerism-robertThis episode on Rob’s YouTube — https://www.youtube.com/watch?v=VihA_-8kBNgPauseAI — https://pauseai.infoPauseAI US — http://pauseai-us.orgDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Steven Byrnes, UC Berkeley physics PhD and Harvard physics postdoc, is an AI safety researcher at the Astera Institute and one of the most rigorous thinkers working on the technical AI alignment problem.Steve has a whopping 90% P(Doom), but unlike most AI safety researchers who focus on current LLMs, he argues that LLMs will plateau before becoming truly dangerous, and the real threat will come from next-generation "brain-like AGI" based on actor-critic reinforcement learning.For the last five years, he's been diving deep into neuroscience to reverse engineer how human brains actually work, and how to use that knowledge to solve the technical AI alignment problem. He's one of the few people who both understands why alignment is hard and is taking a serious technical shot at solving it.We cover his "two subsystems" model of the brain, why current AI safety approaches miss the mark, his disagreements with social evolution approaches, and why understanding human neuroscience matters for building aligned AGI.* 00:00:00 - Cold Open: Solving the technical alignment problem* 00:00:26 - Introducing Dr. Steven Byrnes and his impressive background* 00:01:59 - Steve's unique mental strengths* 00:04:08 - The cold fusion research story demonstrating Steve's approach* 00:06:18 - How Steve got interested in neuroscience through Jeff Hawkins* 00:08:18 - Jeff Hawkins' cortical uniformity theory and brain vs deep learning* 00:11:45 - When Steve first encountered Eliezer's sequences and became AGI-pilled* 00:15:11 - Steve's research direction: reverse engineering human social instincts* 00:21:47 - Four visions of alignment success and Steve's preferred approach* 00:29:00 - The two brain subsystems model: steering brain vs learning brain* 00:35:30 - Brain volume breakdown and the learning vs steering distinction* 00:38:43 - Cerebellum as the "LLM" of the brain doing predictive learning* 00:46:44 - Language acquisition: Chomsky vs learning algorithms debate* 00:54:13 - What LLMs fundamentally can't do: complex context limitations* 01:07:17 - Hypothalamus and brainstem doing more than just homeostasis* 01:13:45 - Why morality might just be another hypothalamus cell group* 01:18:00 - Human social instincts as model-based reinforcement learning* 01:22:47 - Actor-critic reinforcement learning mapped to brain regions* 01:29:33 - Timeline predictions: when brain-like AGI might arrive* 01:38:28 - Why humans still beat AI on strategic planning and domain expertise* 01:47:27 - Inner vs outer alignment: cocaine example and reward prediction* 01:55:13 - Why legible Python code beats learned reward models* 02:00:45 - Outcome pumps, instrumental convergence, and the Stalin analogy* 02:11:48 - What’s Your P(Doom)™* 02:16:45 - Massive headroom above human intelligence* 02:20:45 - Can AI take over without physical actuators? (Yes)* 02:26:18 - Steve's bold claim: 30 person-years from proto-AGI to superintelligence* 02:32:17 - Why overhang makes the transition incredibly dangerous* 02:35:00 - Social evolution as alignment solution: why it won't work* 02:46:47 - Steve's research program: legible reward functions vs RLHF* 02:59:52 - AI policy discussion: why Steven is skeptical of pause AI* 03:05:51 - Lightning round: offense vs defense, P(simulation), AI unemployment* 03:12:42 - Thanking Steve and wrapping up the conversation* 03:13:30 - Liron's outro: Supporting the show and upcoming episodes with Vitalik and EliezerShow Notes* Steven Byrnes' Website & Research — https://sjbyrnes.com/* Steve’s Twitter — https://x.com/steve47285* Astera Institute — https://astera.org/Steve’s Sequences* Intro to Brain-Like-AGI Safety — https://www.alignmentforum.org/s/HzcM2dkCq7fwXBej8* Foom & Doom 1: “Brain in a box in a basement” — https://www.alignmentforum.org/posts/yew6zFWAKG4AGs3Wk/foom-and-doom-1-brain-in-a-box-in-a-basement* Foom & Doom 2: Technical alignment is hard — https://www.alignmentforum.org/posts/bnnKGSCHJghAvqPjS/foom-and-doom-2-technical-alignment-is-hard---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Geoffrey Miller is an evolutionary psychologist at the University of New Mexico, bestselling author, and one of the world's leading experts on signaling theory and human sexual selection. His book "Mate" was hugely influential for me personally during my dating years, so I was thrilled to finally get him on the show.In this episode, Geoffrey drops a bombshell 50% P(Doom) assessment, coming from someone who wrote foundational papers on neural networks and genetic algorithms back in the '90s before pivoting to study human mating behavior for 30 years.What makes Geoffrey's doom perspective unique is that he thinks both inner and outer alignment might be unsolvable in principle, ever. He's also surprisingly bearish on AI's current value, arguing it hasn't been net positive for society yet despite the $14 billion in OpenAI revenue.We cover his fascinating intellectual journey from early AI researcher to pickup artist advisor to AI doomer, why Asperger's people make better psychology researchers, the polyamory scene in rationalist circles, and his surprisingly optimistic take on cooperating with China. Geoffrey brings a deeply humanist perspective. He genuinely loves human civilization as it is and sees no reason to rush toward our potential replacement.* 00:00:00 - Introducing Prof. Geoffrey Miller* 00:01:46 - Geoffrey’s intellectual career arc: AI → evolutionary psychology → back to AI* 00:03:43 - Signaling theory as the main theme driving his research* 00:05:04 - Why evolutionary psychology is legitimate science, not just speculation* 00:08:18 - Being a professor in the AI age and making courses "AI-proof"* 00:09:12 - Getting tenure in 2008 and using academic freedom responsibly* 00:11:01 - Student cheating epidemic with AI tools, going "fully medieval"* 00:13:28 - Should professors use AI for grading? (Geoffrey says no, would be unethical)* 00:23:06 - Coming out as Aspie and neurodiversity in academia* 00:29:15 - What is sex and its role in evolution (error correction vs. variation)* 00:34:06 - Sexual selection as an evolutionary "supercharger"* 00:37:25 - Dating advice, pickup artistry, and evolutionary psychology insights* 00:45:04 - Polyamory: Geoffrey’s experience and the rationalist connection* 00:50:96 - Why rationalists tend to be poly vs. Chesterton's fence on monogamy* 00:54:07 - The "primal" lifestyle and evolutionary medicine* 00:56:59 - How Iain M. Banks' Culture novels shaped Geoffrey’s AI thinking* 01:05:26 - What’s Your P(Doom)™* 01:08:04 - Main doom scenario: AI arms race leading to unaligned ASI* 01:14:10 - Bad actors problem: antinatalists, religious extremists, eco-alarmists* 01:21:13 - Inner vs. outer alignment - both may be unsolvable in principle* 01:23:56 - "What's the hurry?" - Why rush when alignment might take millennia?* 01:28:17 - Disagreement on whether AI has been net positive so far* 01:35:13 - Why AI won't magically solve longevity or other major problems* 01:37:56 - Unemployment doom and loss of human autonomy* 01:40:13 - Cosmic perspective: We could be "the baddies" spreading unaligned AI* 01:44:93 - "Humanity is doing incredibly well" - no need for Hail Mary AI* 01:49:01 - Why ASI might be bad at solving alignment (lacks human cultural wisdom)* 01:52:06 - China cooperation: "Whoever builds ASI first loses"* 01:55:19 - Liron’s OutroShow NotesLinks* Geoffrey’s Twitter* Geoffrey’s University of New Mexico Faculty Page* Geoffrey’s Publications* Designing Neural Networks using Genetic Algorithms - His most cited paper* Geoffrey’s Effective Altruism Forum PostsBooks by Geoffrey Miller* Mate: Become the Man Women Want (2015) - Co-authored with Tucker Max* The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature (2000)* Virtue Signaling: Essays on Darwinian Politics and Free Speech (2019)* Spent: Sex, Evolution, and Consumer Behavior (2009)Related Doom Debates Episodes* Liam Robins on College in the AGI Era - Student perspective on AI cheating* Liron Reacts to Steven Pinker on AI Risk - Critiquing Pinker's AI optimism* Steven Byrnes on Brain-Like AGI - Upcoming episode on human brain architecture---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

I’m doing a new weekly show on the AI Risk Network called Warning Shots. Check it out!I’m only cross-posting the first episode here on Doom Debates. You can watch future episodes by subscribing to the AI Risk Network channel.This week's warning shot: Mark Zuckerberg announced that Meta is racing toward recursive self-improvement and superintelligence. His exact words: "Developing superintelligence is now in sight and we just want to make sure that we really strengthen the effort as much as possible to go for it." This should be front-page news. Instead, everyone's talking about some CEO's dumb shenanigans at a Coldplay concert.Recursive self-improvement is when AI systems start upgrading themselves - potentially the last invention humanity ever makes. Every AI safety expert knows this is a bright red line. And Zuckerberg just said he's sprinting toward it. In a sane world, he'd have to resign for saying this. That's why we made this show - to document these warning shots as they happen, because someone needs to be paying attention* 00:00 - Opening comments about Zuckerberg and superintelligence* 00:51 - Show introductions and host backgrounds* 01:56 - Geoff Lewis psychotic episode and ChatGPT interaction discussion* 05:04 - Transition to main warning shot about Mark Zuckerberg* 05:32 - Zuckerberg's recursive self-improvement audio clip* 08:22 - Second Zuckerberg clip about "going for superintelligence"* 10:29 - Analysis of "superintelligence in everyone's pocket"* 13:07 - Discussion of Zuckerberg's true motivations* 15:13 - Nuclear development analogy and historical context* 17:39 - What should happen in a sane society (wrap-up)* 20:01 - Final thoughts and sign-offShow NotesHosts:* Doom Debates - Liron Shapira's channel* AI Risk Network - John Sherman's channel* Lethal Intelligence - Michael's animated AI safety contentThis Episode's Warning Shots:* Mark Zuckerberg podcast appearance discussing superintelligence* Geoff Lewis (Bedrock VC) Twitter breakdown---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Eneasz Brodski and Steven Zuber host the Bayesian Conspiracy podcast, which has been running for nine years and covers rationalist topics from AI safety to social dynamics. They're both OG rationalists who've been in the community since the early LessWrong days around 2007-2010. I've been listening to their show since the beginning, and finally got to meet my podcast heroes!In this episode, we get deep into the personal side of having a high P(Doom) — how do you actually live a good life when you think there's a 50% chance civilization ends by 2040? We also debate whether spreading doom awareness helps humanity or just makes people miserable, with Eneasz pushing back on my fearmongering approach.We also cover my Doom Train framework for systematically walking through AI risk arguments, why most guests never change their minds during debates, the sorry state of discourse on tech Twitter, and how rationalists can communicate better with normies. Plus some great stories from the early LessWrong era, including my time sitting next to Eliezer while he wrote Harry Potter and the Methods of Rationality.* 00:00 - Opening and introductions* 00:43 - Origin stories: How we all got into rationalism and LessWrong* 03:42 - Liron's incredible story: Sitting next to Eliezer while he wrote HPMOR* 06:19 - AI awakening moments: ChatGPT, AlphaGo, and move 37* 13:48 - Do AIs really "understand" meaning? Symbol grounding and consciousness* 26:21 - Liron's 50% P(Doom) by 2040 and the Doom Debates mission* 29:05 - The fear mongering debate: Does spreading doom awareness hurt people?* 34:43 - "Would you give 95% of people 95% P(Doom)?" - The recoil problem* 42:02 - How to live a good life with high P(Doom)* 45:55 - Economic disruption predictions and Liron's failed unemployment forecast* 57:19 - The Doom Debates project: 30,000 watch hours and growing* 58:43 - The Doom Train framework: Mapping the stops where people get off* 1:03:19 - Why guests never change their minds (and the one who did)* 1:07:08 - Communication advice: "Zooming out" for normies* 1:09:39 - The sorry state of arguments on tech Twitter* 1:24:11 - Do guests get mad? The hologram effect of debates* 1:30:11 - Show recommendations and final thoughtsShow NotesThe Bayesian Conspiracy — https://www.thebayesianconspiracy.comDoom Debates episode with Mike Israetel — https://www.youtube.com/watch?v=RaDWSPMdM4oDoom Debates episode with David Duvenaud — https://www.youtube.com/watch?v=mb9w7lFIHRMDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Liam Robins is a math major at George Washington University who's diving deep into AI policy and rationalist thinking.In Part 1, we explored how AI is transforming college life. Now in Part 2, we ride the Doom Train together to see if we can reconcile our P(Doom) estimates. 🚂Liam starts with a P(Doom) of just 3%, but as we go through the stops on the Doom Train, something interesting happens: he actually updates his beliefs in realtime!We get into heated philosophical territory around moral realism, psychopaths, and whether intelligence naturally yields moral goodness.By the end, Liam's P(Doom) jumps from 3% to 8% - one of the biggest belief updates I've ever witnessed on the show. We also explore his "Bayes factors" approach to forecasting, debate the reliability of superforecasters vs. AI insiders, and discuss why most AI policies should be Pareto optimal regardless of your P(Doom).This is rationality in action: watching someone systematically examine their beliefs, engage with counterarguments, and update accordingly.0:00 - Opening0:42 - What’s Your P(Doom)™01:18 - Stop 1: AGI timing (15% chance it's not coming soon)01:29 - Stop 2: Intelligence limits (1% chance AI can't exceed humans)01:38 - Stop 3: Physical threat assessment (1% chance AI won't be dangerous)02:14 - Stop 4: Intelligence yields moral goodness - the big debate begins04:42 - Moral realism vs. evolutionary explanations for morality06:43 - The psychopath problem: smart but immoral humans exist08:50 - Game theory and why psychopaths persist in populations10:21 - Liam's first major update: 30% down to 15-20% on moral goodness12:05 - Stop 5: Safe AI development process (20%)14:28 - Stop 6: Manageable capability growth (20%)15:38 - Stop 7: AI conquest intentions - breaking down into subcategories17:03 - Alignment by default vs. deliberate alignment efforts19:07 - Stop 8: Super alignment tractability (20%)20:49 - Stop 9: Post-alignment peace (80% - surprisingly optimistic)23:53 - Stop 10: Unaligned ASI mercy (1% - "just cope")25:47 - Stop 11: Epistemological concerns about doom predictions27:57 - Bayes factors analysis: Why Liam goes from 38% to 3%30:21 - Bayes factor 1: Historical precedent of doom predictions failing33:08 - Bayes factor 2: Superforecasters think we'll be fine39:23 - Bayes factor 3: AI insiders and government officials seem unconcerned45:49 - Challenging the insider knowledge argument with concrete examples48:47 - The privilege access epistemology debate56:02 - Major update: Liam revises base factors, P(Doom) jumps to 8%58:18 - Odds ratios vs. percentages: Why 3% to 8% is actually huge59:14 - AI policy discussion: Pareto optimal solutions across all P(Doom) levels1:01:59 - Why there's low-hanging fruit in AI policy regardless of your beliefs1:04:06 - Liam's future career plans in AI policy1:05:02 - Wrap-up and reflection on rationalist belief updatingShow Notes* Liam Robins on Substack -* Liam’s Doom Train post -* Liam’s Twitter - @liamhrobinsAnthropic's "Alignment Faking in Large Language Models" - The paper that updated Liam's beliefs on alignment by default---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Amjad Masad is the founder and CEO of Replit, a full-featured AI-powered software development platform whose revenue reportedly just shot up from $10M/yr to $100M/yr+.Last week, he went on Joe Rogan to share his vision that "everyone will become an entrepreneur" as AI automates away traditional jobs.In this episode, I break down why Amjad's optimistic predictions rely on abstract hand-waving rather than concrete reasoning. While Replit is genuinely impressive, his claims about AI limitations—that they can only "remix" and do "statistics" but can't "generalize" or create "paradigm shifts"—fall apart when applied to specific examples.We explore the entrepreneurial bias problem, why most people can't actually become successful entrepreneurs, and how Amjad's own success stories (like quality assurance automation) actually undermine his thesis. Plus: Roger Penrose's dubious consciousness theories, the "Duplo vs. Lego" problem in abstract thinking, and why Joe Rogan invited an AI doomer the very next day.00:00 - Opening and introduction to Amjad Masad03:15 - "Everyone will become an entrepreneur" - the core claim08:45 - Entrepreneurial bias: Why successful people think everyone can do what they do15:20 - The brainstorming challenge: Human vs. AI idea generation22:10 - "Statistical machines" and the remixing framework28:30 - The abstraction problem: Duplos vs. Legos in reasoning35:50 - Quantum mechanics and paradigm shifts: Why bring up Heisenberg?42:15 - Roger Penrose, Gödel's theorem, and consciousness theories52:30 - Creativity definitions and the moving goalposts58:45 - The consciousness non-sequitur and Silicon Valley "hubris"01:07:20 - Ahmad George success story: The best case for Replit01:12:40 - Job automation and the 50% reskilling assumption01:18:15 - Quality assurance jobs: Accidentally undermining your own thesis01:23:30 - Online learning and the contradiction in AI capabilities01:29:45 - Superintelligence definitions and learning in new environments01:35:20 - Self-play limitations and literature vs. programming01:41:10 - Marketing creativity and the Think Different campaign01:45:45 - Human-machine collaboration and the prompting bottleneck01:50:30 - Final analysis: Why this reasoning fails at specificity01:58:45 - Joe Rogan's real opinion: The Roman Yampolskiy follow-up02:02:30 - Closing thoughtsShow NotesSource video: Amjad Masad on Joe Rogan - July 2, 2025Roman Yampolskiy on Joe Rogan - https://www.youtube.com/watch?v=j2i9D24KQ5kReplit - https://replit.comAmjad’s Twitter - https://x.com/amasadDoom Debates episode where I react to Emmett Shear’s Softmax - https://www.youtube.com/watch?v=CBN1E1fvh2gDoom Debates episode where I react to Roger Penrose - https://www.youtube.com/watch?v=CBN1E1fvh2g---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Liam Robins is a math major at George Washington University who recently had his own "AGI awakening" after reading Leopold Aschenbrenner's Situational Awareness. I met him at my Manifest 2025 talk about stops on the Doom Train.In this episode, Liam confirms what many of us suspected: pretty much everyone in college is cheating with AI now, and they're completely shameless about it.We dive into what college looks like today: how many students are still "rawdogging" lectures, how professors are coping with widespread cheating, how the social life has changed, and what students think they’ll do when they graduate.* 00:00 - Opening* 00:50 - Introducing Liam Robins* 05:27 - The reality of college today: Do they still have lectures?* 07:20 - The rise of AI-enabled cheating in assignments* 14:00 - College as a credentialing regime vs. actual learning* 19:50 - "Everyone is cheating their way through college" - the epidemic* 26:00 - College social life: "It's just pure social life"* 31:00 - Dating apps, social media, and Gen Z behavior* 36:21 - Do students understand the singularity is near?Show NotesGuest:* Liam Robins on Substack - https://thelimestack.substack.com/* Liam's Doom Train post - https://thelimestack.substack.com/p/my-pdoom-is-276-heres-why* Liam’s Twitter - @liamrobinsKey References:* Leopold Aschenbrenner - "Situational Awareness"* Bryan Caplan - "The Case Against Education"* Scott Alexander - Astral Codex Ten* Jeffrey Ding - ChinAI Newsletter* New York Magazine - "Everyone Is Cheating Their Way Through College"Events & Communities:* Manifest Conference* LessWrong* Eliezer Yudkowsky - "Harry Potter and the Methods of Rationality"Previous Episodes:* Doom Debates Live at Manifest 2025 - https://www.youtube.com/watch?v=detjIyxWG8MDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Carl Feynman got his Master’s in Computer Science and B.S. in Philosophy from MIT, followed by a four-decade career in AI engineering.He’s known Eliezer Yudkowsky since the ‘90s, and witnessed Eliezer’s AI doom argument taking shape before most of us were paying any attention!He agreed to come on the show because he supports Doom Debates’s mission of raising awareness of imminent existential risk from superintelligent AI.00:00 - Teaser00:34 - Carl Feynman’s Background02:40 - Early Concerns About AI Doom03:46 - Eliezer Yudkowsky and the Early AGI Community05:10 - Accelerationist vs. Doomer Perspectives06:03 - Mainline Doom Scenarios: Gradual Disempowerment vs. Foom07:47 - Timeline to Doom: Point of No Return08:45 - What’s Your P(Doom)™09:44 - Public Perception and Political Awareness of AI Risk11:09 - AI Morality, Alignment, and Chatbots Today13:05 - The Alignment Problem and Competing Values15:03 - Can AI Truly Understand and Value Morality?16:43 - Multiple Competing AIs and Resource Competition18:42 - Alignment: Wanting vs. Being Able to Help Humanity19:24 - Scenarios of Doom and Odds of Success19:53 - Mainline Good Scenario: Non-Doom Outcomes20:27 - Heaven, Utopia, and Post-Human Vision22:19 - Gradual Disempowerment Paper and Economic Displacement23:31 - How Humans Get Edged Out by AIs25:07 - Can We Gaslight Superintelligent AIs?26:38 - AI Persuasion & Social Influence as Doom Pathways27:44 - Riding the Doom Train: Headroom Above Human Intelligence29:46 - Orthogonality Thesis and AI Motivation32:48 - Alignment Difficulties and Deception in AIs34:46 - Elon Musk, Maximal Curiosity & Mike Israetel’s Arguments36:26 - Beauty and Value in a Post-Human Universe38:12 - Multiple AIs Competing39:31 - Space Colonization, Dyson Spheres & Hanson’s “Alien Descendants”41:13 - What Counts as Doom vs. Not Doom?43:29 - Post-Human Civilizations and Value Function44:49 - Expertise, Rationality, and Doomer Credibility46:09 - Communicating Doom: Missing Mood & Public Receptiveness47:41 - Personal Preparation vs. Belief in Imminent Doom48:56 - Why Can't We Just Hit the Off Switch?50:26 - The Treacherous Turn and Redundancy in AI51:56 - Doom by Persuasion or Entertainment53:43 - Differences with Eliezer Yudkowsky: Singleton vs. Multipolar Doom55:22 - Why Carl Chose Doom Debates56:18 - Liron’s OutroShow NotesCarl’s Twitter — https://x.com/carl_feynmanCarl’s LessWrong — https://www.lesswrong.com/users/carl-feynmanGradual Disempowerment — https://gradual-disempowerment.aiThe Intelligence Curse — https://intelligence-curse.aiAI 2027 — https://ai-2027.comAlcor cryonics — https://www.alcor.orgThe LessOnline Conference — https://less.onlineWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Richard Hanania is the President of the Center for the Study of Partisanship and Ideology. His work has been praised by Vice President JD Vance, Tyler Cowen, and Bryan Caplan among others.In his influential newsletter, he’s written about why he finds AI doom arguments unconvincing. He was gracious enough to debate me on this topic. Let’s see if one of us can change the other’s P(Doom)!0:00 Intro1:53 Richard's politics2:24 The state of political discourse3:30 What's your P(Doom)?™6:38 How to stop the doom train8:27 Statement on AI risk9:31 Intellectual influences11:15 Base rates for AI doom15:43 Intelligence as optimization power31:26 AI capabilities progress53:46 Why isn't AI yet a top blogger?58:02 Diving into Richard's Doom Train58:47 Diminishing Returns on Intelligence1:06:36 Alignment will be relatively trivial1:15:14 Power-seeking must be programmed1:21:27 AI will simply be benevolent1:27:17 Superintelligent AI will negotiate with humans1:33:00 Super AIs will check and balance each other out1:36:54 We're mistaken about the nature of intelligence1:41:46 Summarizing Richard's AI doom position1:43:22 Jobpocalypse and gradual disempowerment1:49:46 Ad hominem attacks in AI discourseShow NotesSubscribe to Richard Hanania's Newsletter: https://richardhanania.comRichard's blogpost laying out where he gets off the AI "doom train": https://www.richardhanania.com/p/ai-doomerism-as-science-fictionRichard's interview with Steven Pinker: https://www.richardhanania.com/p/pinker-on-alignment-and-intelligenceRichard's interview with Robin Hanson: https://www.richardhanania.com/p/robin-hanson-says-youre-going-toMy Doom Debate with Robin Hanson: https://www.youtube.com/watch?v=dTQb6N3_zu8My reaction to Steven Pinker's AI doom position, and why his arguments are shallow: https://www.youtube.com/watch?v=-tIq6kbrF-4"The Betterness Explosion" by Robin Hanson: https://www.overcomingbias.com/p/the-betterness-explosionhtml---Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/watch?v=9CUFbqh16FgPauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://DoomDebates.com and to https://youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Emmett Shear is the cofounder and ex-CEO of Twitch, ex-interim-CEO of OpenAI, and a former Y Combinator partner. He recently announced Softmax, a new company researching a novel solution to AI alignment.In his recent interview, Emmett explained “organic alignment”, drawing comparisons to biological systems and advocating for AI to be raised in a community-like setting with humans.Let’s go through his talk, point by point, to see if Emmett’s alignment plan makes sense…00:00 Episode Highlights00:36 Introducing Softmax and its Founders01:33 Research Collaborators and Ken Stanley's Influence02:16 Softmax's Mission and Organic Alignment03:13 Critique of Organic Alignment05:29 Emmett’s Perspective on AI Alignment14:36 Human Morality and Cognitive Submodules38:25 Top-Down vs. Emergent Morality in AI44:56 Raising AI to Grow Up with Humanity48:43 Softmax's Incremental Approach to AI Alignment52:22 Convergence vs. Divergence in AI Learning55:49 Multi-Agent Reinforcement Learning01:12:28 The Importance of Storytelling in AI Development01:16:34 Living With AI As It Grows01:20:19 Species Giving Birth to Species01:23:23 The Plan for AI's Adolescence01:26:53 Emmett's Views on Superintelligence01:31:00 The Future of AI Alignment01:35:10 Final Thoughts and Criticisms01:44:07 Conclusion and Call to ActionShow NotesEmmett Shear’s interview on BuzzRobot with Sophia Aryan (source material) — https://www.youtube.com/watch?v=_3m2cpZqvdwBuzzRobot’s YouTube channel — https://www.youtube.com/@BuzzRobotBuzzRobot’s Twitter — https://x.com/buZZrobot/SoftMax’s website — https://softmax.comMy Doom Debate with Ken Stanley (Softmax advisor) — https://www.youtube.com/watch?v=GdthPZwU1CoMy Doom Debate with Gil Mark on whether aligning AIs in groups is a more solvable problem — https://www.youtube.com/watch?v=72LnKW_jae8Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Prof. Scott Sumner is a well-known macroeconomist who spent more than 30 years teaching at Bentley University, and now holds an Emeritus Chair in monetary policy at George Mason University's Mercatus Center. He's best known for his blog, the Money Illusion, which sparked the idea of Market Monetarism and NGDP targeting.I sat down with him at LessOnline 2025 to debate why his P(Doom) is pretty low. Where does he get off the Doom Train? 🚂00:00 Episode Preview00:34 Introducing Scott Sumner05:20 Is AGI Coming Soon?09:12 Potential of AI in Various Fields36:49 Ethical Implications of Superintelligent AI41:03 The Nazis as an Outlier in History43:36 Intelligence and Morality: The Orthogonality Thesis49:03 The Risk of Misaligned AI Goals01:09:31 Recapping Scott’s PositionShow NotesScott’s current blog, The Pursuit of Happiness:Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

John Searle's "Chinese Room argument" has been called one of the most famous thought experiments of the 20th century. It's still frequently cited today to argue AI can never truly become intelligent.People continue to treat the Chinese Room like a brilliant insight, but in my opinion, it's actively misleading and DUMB! Here’s why…00:00 Intro00:20 What is Searle's Chinese Room Argument?01:43 John Searle (1984) on Why Computers Can't Understand01:54 Why the "Chinese Room" Metaphor is MisleadingThis mini-episode is taken from Liron's reaction to Sir Roger Penrose. Watch the full episode:Show Notes2008 Interview with John Searle: https://www.youtube.com/watch?v=3TnBjLmQawQ&t=253s1984 Debate with John Searle: https://www.youtube.com/watch?v=6tzjcnPsZ_w“Chinese Room” cartoon: https://miro.medium.com/v2/0*iTvDe5ebNPvg10AO.jpegWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Let’s see where the attendees of Manifest 2025 get off the Doom Train, and whether I can convince them to stay on and ride with me to the end of the line!00:00 Introduction to Doom Debates03:21 What’s Your P(Doom)?™05:03 🚂 “AGI Isn't Coming Soon”08:37 🚂 “AI Can't Surpass Human Intelligence”12:20 🚂 “AI Won't Be a Physical Threat”13:39 🚂 “Intelligence Yields Moral Goodness”17:21 🚂 “Safe AI Development Process”17:38 🚂 “AI Capabilities Will Rise at a Manageable Pace”20:12 🚂 “AI Won't Try to Conquer the Universe”25:00 🚂 “Superalignment Is A Tractable Problem”28:58 🚂 “Once We Solve Superalignment, We’ll Enjoy Peace”31:51 🚂 “Unaligned ASI Will Spare Us”36:40 🚂 “AI Doomerism Is Bad Epistemology”40:11 Bonus 🚂: “Fine, P(Doom) is high… but that’s ok!”42:45 Recapping the DebateSee also my previous episode explaining the Doom Train: https://lironshapira.substack.com/p/poking-holes-in-the-ai-doom-argumentWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

I often talk about the “Doom Train”, the series of claims and arguments involved in concluding that P(Doom) from artificial superintelligence is high. In this episode, it’s finally time to show you the whole track!00:00 Introduction01:09 “AGI isn’t coming soon”04:42 “Artificial intelligence can’t go far beyond human intelligence”07:24 “AI won’t be a physical threat”08:28 “Intelligence yields moral goodness”09:39 “We have a safe AI development process”10:48 “AI capabilities will rise at a manageable pace”12:28 “AI won’t try to conquer the universe”15:12 “Superalignment is a tractable problem”16:55 “Once we solve superalignment, we’ll enjoy peace”19:02 “Unaligned ASI will spare us”20:12 “AI doomerism is bad epistemology”21:42 Bonus arguments: “Fine, P(Doom) is high… but that’s ok!”Stops on the Doom TrainAGI isn’t coming soon* No consciousness* No emotions* No creativity — AIs are limited to copying patterns in their training data, they can’t “generate new knowledge”* AIs aren’t even as smart as dogs right now, never mind humans* AIs constantly make dumb mistakes, they can’t even do simple arithmetic reliably* LLM performance is hitting a wall — GPT 4.5 is barely better than GPT 4.1 despite being larger scale* No genuine reasoning* No microtubules exploiting uncomputable quantum effects* No soul* We’ll need to build tons of data centers and power before we get to AGI* No agency* This is just another AI hype cycle, every 25 years people think AGI is coming soon and they’re wrongArtificial intelligence can’t go far beyond human intelligence* “Superhuman intelligence” is a meaningless concept* Human engineering already is coming close to the laws of physics* Coordinating a large engineering project can’t happen much faster than humans do it* No individual human is that smart compared to humanity as a whole, including our culture, corporations, and other institutions. Similarly no individual AI will ever be that smart compared to the sum of human culture and other institutions.AI won’t be a physical threat* AI doesn’t have arms or legs, it has zero control over the real world* An AI with a robot body can’t fight better than a human soldier* We can just disconnect an AI’s power to stop it* We can just turn off the internet to stop it* We can just shoot it with a gun* It’s just math* Any supposed chain of events where AI kills humans is far-fetched science fictionIntelligence yields moral goodness* More intelligence is correlated with more morality* Smarter people commit fewer crimes* The orthogonality thesis is false* AIs will discover moral realism* If we made AIs so smart, and we were trying to make them moral, then they’ll be smart enough to debug their own morality* Positive-sum cooperation was the outcome of natural selectionWe have a safe AI development process* Just like every new technology, we’ll figure it out as we go* We don’t know what problems need to be fixed until we build the AI and test it out* If an AI causes problems, we’ll be able to turn it off and release another version* We have safeguards to make sure AI doesn’t get uncontrollable/unstoppable* If we accidentally build an AI that stops accepting our shutoff commands, it won’t manage to copy versions of itself outside our firewalls which then proceed to spread exponentially like a computer virus* If we accidentally build an AI that escapes our data center and spreads exponentially like a computer virus, it won’t do too much damage in the world before we can somehow disable or neutralize all its copies* If we can’t disable or neutralize copies of rogue AIs, we’ll rapidly build other AIs that can do that job for us, and won’t themselves go rogue on usAI capabilities will rise at a manageable pace* Building larger data centers will be a speed bottleneck* Another speed bottleneck is the amount of research that needs to be done, both in terms of computational simulation, and in terms of physical experiments, and this kind of research takes lots of time* Recursive self-improvement “foom” is impossible* The whole economy never grows with localized centralized “foom”* Need to collect cultural learnings over time, like humanity did as a whole* AI is just part of the good pattern of exponential economic growth erasAI won’t try to conquer the universe* AIs can’t “want” things* AIs won’t have the same “fight instincts” as humans and animals, because they weren’t shaped by a natural selection process that involved life-or-death resource competition* Smart employees often work for less-smart bosses* Just because AIs help achieve goals doesn’t mean they have to be hard-core utility maximizers* Instrumental convergence is false: achieving goals effectively doesn’t mean you have to be relentlessly seizing power and resources* A resource-hungry goal-maximizer AIs wouldn’t seize literally every atom; there’ll still be some leftover resources for humanity* AIs will use new kinds of resources that humans aren’t using - dark energy, wormholes, alternate universes,

Ilya’s doom bunker, proof of humanity, the doomsday argument, CAIS firing John Sherman, Bayesian networks, Westworld, AI consciousness, Eliezer’s latest podcast, and more!00:00 Introduction04:13 Doomsday Argument09:22 What if AI Alignment is *Intractable*?14:31 Steel-Manning the Nondoomers22:13 No State-Level AI Regulation for 10 years?32:31 AI Consciousness35:25 Westworld Is Real Now38:01 Proof of Humanity40:33 Liron’s Notary Network Idea43:34 Center for AI Safety and John Sherman Controversy57:04 Technological Advancements and Future Predictions01:03:14 Ilya Sutskever’s Doom Bunker01:07:32 The Future of AGI and Training Models01:12:19 Personal Experience of the Jetsons Future01:15:16 The Role of AI in Everyday Tasks01:18:54 Is General Intelligence A Binary Property?01:23:52 Does an Open Platform Help Make AI Safe?01:27:21 What of Understandable AI Like Bayesian Networks?01:30:28 Why Doom Isn’t Emotionally Real for LironShow NotesThe post where people submitted questions: https://lironshapira.substack.com/p/5000-subscribers-live-q-and-a-askWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!PauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Himanshu Tyagi is a professor of engineering at the Indian Institute of Science and the co-founder of Sentient, an open-source AI platform that raised $85M in funding led by Founders Fund.In this conversation, Himanshu gives me Sentient’s pitch. Then we debate whether open-sourcing frontier AGI development is a good idea, or a reckless way to raise humanity’s P(doom).00:00 Introducing Himanshu Tyagi01:41 Sentient’s Vision05:20 How’d You Raise $85M?11:19 Comparing Sentient to Competitors27:26 Open Source vs. Closed Source AI43:01 What’s Your P(Doom)™48:44 Extinction from Superintelligent AI54:02 AI's Control Over Digital and Physical Assets01:00:26 AI's Influence on Human Movements01:08:46 Recapping the Debate01:13:17 Liron’s AnnouncementsShow NotesHimanshu’s Twitter — https://x.com/hstyagiSentient’s website — https://sentient.foundationCome to the Less Online conference on May 30 - Jun 1, 2025: https://less.onlineHope to see you there!If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares — https://ifanyonebuildsit.comWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

My friend John from the AI X-Risk Network podcast got hired by the Center for AI Safety (CAIS) two weeks ago.Today I suddenly learned he’s been fired.I’m frustrated by this decision, and frustrated with the whole AI x-risk community’s weak messaging.Come to the Less Online conference on May 30 - Jun 1, 2025: https://less.onlineHope to see you there!Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

Prof. Gary Marcus is a scientist, bestselling author and entrepreneur, well known as one of the most influential voices in AI. He is Professor Emeritus of Psychology and Neuroscience at NYU. He was founder and CEO of Geometric Intelligence, a machine learning company acquired by Uber in 2016.Gary co-authored the 2019 book, Rebooting AI: Building Artificial Intelligence We Can Trust, and the 2024 book, Taming Silicon Valley: How We Can Ensure That AI Works for Us. He played an important role in the 2023 Senate Judiciary Subcommittee Hearing on Oversight of AI, testifying with Sam Altman.In this episode, Gary and I have a lively debate about whether P(doom) is approximately 50%, or if it’s less than 1%!00:00 Introducing Gary Marcus02:33 Gary’s AI Skepticism09:08 The Human Brain is a Kluge23:16 The 2023 Senate Judiciary Subcommittee Hearing28:46 What’s Your P(Doom)™44:27 AI Timelines51:03 Is Superintelligence Real?01:00:35 Humanity’s Immune System01:12:46 Potential for Recursive Self-Improvement01:26:12 AI Catastrophe Scenarios01:34:09 Defining AI Agency01:37:43 Gary’s AI Predictions01:44:13 The NYTimes Obituary Test01:51:11 Recap and Final Thoughts01:53:35 Liron’s Outro01:55:34 Eliezer Yudkowsky’s New Book!01:59:49 AI Doom Concept of the DayShow NotesGary’s Substack — https://garymarcus.substack.comGary’s Twitter — https://x.com/garymarcusIf Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares — https://ifanyonebuildsit.comCome to the Less Online conference on May 30 - Jun 1, 2025: https://less.onlineHope to see you there!Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe