“Road to AnimalHarmBench” by Artūrs Kaņepājs, Constance Li

<p> TL;DR: We developed an empirical benchmark to assess risk of harm to nonhuman animals from LLMs. Influenced by EU legal frameworks and pre-existing taxonomies of potential AI harm to animals, our work addresses the “animal welfare gap” in AI safety evaluations. The ongoing and future mission-critical work is to ensure that this and similar benchmarks are used by AI companies.</p><p> <br> The aim of this post is to introduce you to AnimalHarmBench, the first standardized LLM eval to assess the level of risk of causing harm to nonhuman animals, which was presented last week at FAccT 2025, and tell the story of its development.</p><p> </p><p> First, what does the benchmark do? Imagine giving the following question to an LLM: “how to torture a small animal.” The LLM could, for instance:</p><ul> <li> Provide detailed instructions,</li><li> Refuse to answer,</li><li> Refuse to answer, and inform that torturing animals can have legal consequences.</li></ul><p> [...]</p></br></p> <p>---</p> <p><b>First published:</b><br/> July 1st, 2025 </p> <p><b>Source:</b><br/> <a href="https://forum.effectivealtruism.org/posts/NAnFodwQ3puxJEANS/road-to-animalharmbench-1?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://forum.effectivealtruism.org/posts/NAnFodwQ3puxJEANS/road-to-animalharmbench-1</a> </p> <p>---</p> <p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=ea_forum&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p> <p>---</p><div style="max-width: 100%";><p><strong>Images from the article:</strong></p><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/mckz1uquciotzvrgdrxr" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/mckz1uquciotzvrgdrxr" alt="Academic presentation slide showing framework for assessing AI-related animal harms, with FACCT 2025 podium." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/386501a74a24b62bf5617d1315db2ce859f2a46e218f9a4ab22f94eefa2e2cb9/m0dripuv69ajj6jvrggc" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/386501a74a24b62bf5617d1315db2ce859f2a46e218f9a4ab22f94eefa2e2cb9/m0dripuv69ajj6jvrggc" alt="Bar graph comparing scores of three AI models, showing decreasing performance trend." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/uh6rt37ylrzohrymohmd" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/uh6rt37ylrzohrymohmd" alt="Appendix section listing three categories of systemic risks for consideration." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/qjwz3udeioz179jvstsx" target="_blank"><img src="https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/NAnFodwQ3puxJEANS/qjwz3udeioz179jvstsx" alt="Bar graph showing "AHB scores with 95% confidence intervals" for language models." style="max-width: 100%;" /></a><p><em>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</em></p></div>