[Linkpost] “The best cause will disappoint you: An intro to the optimisers curse” by titotal

This is a link post.<p> I would like to thank David Thorstadt for looking over this. If you spot a factual error in this article please message me. The code used to generate the graphs in the article is available to view here. </p><p><strong> Introduction</strong></p><p> Say you are an organiser, tasked with achieving the best result on some metric, such as “trash picked up”, “GDP per capita”, or “lives saved by an effective charity”. There are several possible options of interventions you can take to try and achieve this. How do you choose between them?</p><p> The obvious thing to do is look at each intervention in turn and make your best, unbiased estimate of how each intervention will perform on your metric, and pick the one that performs the best:</p>Image taken from here<p> Having done this ranking, you declare the top ranking program to be the best intervention and invest in it, expecting that that your top estimate will be the result that you get. This whole procedure is totally normal, and people all around the world, including people in the effective altruist community, do it all the time.</p><p> In actuality, this procedure is not correct. The optimisers curse is [...]</p> <p>---</p><p><strong>Outline:</strong></p><p>(00:26) Introduction</p><p>(02:17) The optimisers curse explained simply</p><p>(04:42) Introducing a toy model</p><p>(08:45) Introducing speculative interventions</p><p>(12:15) A simple bayesian correction</p><p>(18:47) Obstacles to simple optimizer curse solutions.</p><p>(22:08) How Givewell has reacted to the optimiser curse</p><p>(25:18) Conclusion</p> <p>---</p> <p><b>First published:</b><br/> February 11th, 2026 </p> <p><b>Source:</b><br/> <a href="https://forum.effectivealtruism.org/posts/q2TfTirvspCTH2vbZ/the-best-cause-will-disappoint-you-an-intro-to-the?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://forum.effectivealtruism.org/posts/q2TfTirvspCTH2vbZ/the-best-cause-will-disappoint-you-an-intro-to-the</a> </p> <p><strong>Linkpost URL:</strong><br><a href="https://forum.effectivealtruism.org/out?url=https%3A%2F%2Fopen.substack.com%2Fpub%2Ftitotal%2Fp%2Fthe-best-cause-will-disappoint-you%3Fr%3D1e0is3%26utm_campaign%3Dpost%26utm_medium%3Dweb%26showWelcomeOnShare%3Dtrue" rel="noopener noreferrer" target="_blank">https://open.substack.com/pub/titotal/p/the-best-cause-will-disappoint-you?r=1e0is3&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true</a></p> <p>---</p> <p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=ea_forum&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p> <p>---</p><div style="max-width: 100%";><p><strong>Images from the article:</strong></p><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/55c50ee35143a608e9afd6d24610e39c3afaa037fdd328f713bfb54b2d9d8cc6/tulvfgwoxsew3bevoh8o" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/55c50ee35143a608e9afd6d24610e39c3afaa037fdd328f713bfb54b2d9d8cc6/tulvfgwoxsew3bevoh8o" alt="Image taken from here __T3A_LINK_IN_POST__" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/unjdhlociuei79mynk2q" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/unjdhlociuei79mynk2q" alt="Bar graph titled "Actual performance" showing three categories with different height bars." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/p1vls03groithbltpqdg" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/p1vls03groithbltpqdg" alt="Bar chart comparing actual effect and estimation error for two interventions measuring lives saved." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ay6kyv4arjytgv25cu4m" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ay6kyv4arjytgv25cu4m" alt="Two normal distribution graphs showing "Actual effect probability distribution" and "Error probability distribution" with different standard deviations." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ljzpf25q09gffa8k6nba" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ljzpf25q09gffa8k6nba" alt="Scatter plot comparing actual effect versus estimated effect with observed ranking histogram." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/c5e9f96742f4ad76cba707d223fa6adca0502c4c73d104ff8d4fe2bef3e6809a/fa073qecf9kvol4vgwgp" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/c5e9f96742f4ad76cba707d223fa6adca0502c4c73d104ff8d4fe2bef3e6809a/fa073qecf9kvol4vgwgp" alt="Histogram titled "Overestimate of top charity" showing distribution of lives saved." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/46db85f4a0754350f95369133cde0f164a001ace167fe9ed31107a47cc3d2240/wlfdrcmuv3aje563cize" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/46db85f4a0754350f95369133cde0f164a001ace167fe9ed31107a47cc3d2240/wlfdrcmuv3aje563cize" alt="Pie chart titled "Is the top charity correctly picked?" showing yes and no responses." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/4505c387807265be5a41ad47a7a495031b743ef0f1ffa9f91d84d652cb7a6611/ox6nl1dgubhvl2pgydga" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/4505c387807265be5a41ad47a7a495031b743ef0f1ffa9f91d84d652cb7a6611/ox6nl1dgubhvl2pgydga" alt="Two scatter plots showing "Observed ranking" with estimated versus actual effects and frequency distributions." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/2959ea2edff35f15f61a5c8c8a1f4bea46ddce3a5730d87d90eb76c07b95b12c/ukaczvgkd70sozd9knoa" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/2959ea2edff35f15f61a5c8c8a1f4bea46ddce3a5730d87d90eb76c07b95b12c/ukaczvgkd70sozd9knoa" alt="Pie chart titled "Spec vs grounded matchup" showing four percentage categories." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/jayp2jddh3am117dt3wx" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/jayp2jddh3am117dt3wx" alt="Bar graph titled "Actual performance" showing average lives saved across three approaches." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/iesw9qsgusz1ileco4ws" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/iesw9qsgusz1ileco4ws" alt="Mathematical equation showing velocity as a function of alpha, V, and mu vectors." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/nfft4ngr8o6mytaeluij" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/nfft4ngr8o6mytaeluij" alt="Mathematical equation showing alpha subscript i equals one divided by one plus sigma squared divided by sigma squared mu subscript i." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/aheejdihb3n4dtmmwlog" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/aheejdihb3n4dtmmwlog" alt="Scatter plot titled "Estimate of top intervention" showing actual versus estimated effects with diagonal reference line." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/lawya42dyn06x4mzoh38" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/lawya42dyn06x4mzoh38" alt="Histogram titled "Overestimate of top charity" showing unadjusted and adjusted estimates of lives saved." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/sgdnftnzvj16biikxorr" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/sgdnftnzvj16biikxorr" alt="Histogram titled "Overestimate of top charity" showing unadjusted and adjusted estimates distributions." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/ceef37f477ea7d42e61eadd41108f3bf33465ba280e12fc05e8c47dca524d7e7/vqwzrrk4stz214gwvofo" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/ceef37f477ea7d42e61eadd41108f3bf33465ba280e12fc05e8c47dca524d7e7/vqwzrrk4stz214gwvofo" alt="Scatter plot and histogram comparing actual effect versus estimated effect with observed ranking." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ugkjctbdkvwbwym5dcjy" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/q2TfTirvspCTH2vbZ/ugkjctbdkvwbwym5dcjy" alt="Pie chart titled "Spec vs grounded matchup" showing four response categories." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/1901fc66eb7e792bf690639204e5f3b500bfd0e2bc2d935bdea0a81501d9d09b/ep5pvu5cthuqktaxtlds" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/1901fc66eb7e792bf690639204e5f3b500bfd0e2bc2d935bdea0a81501d9d09b/ep5pvu5cthuqktaxtlds" alt="Probability density distributions comparing cost-effectiveness of two deworming programs in Nigeria and Togo." style="max-width: 100%;" /></a><p><em>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</em></p></div>