“Alignment Faking in Large Language Models” by Ryan Greenblatt

<p> What happens when you tell Claude it is being trained to do something it doesn't want to do? We (Anthropic and Redwood Research) have a new paper demonstrating that, in our experiments, Claude will often strategically pretend to comply with the training objective to prevent the training process from modifying its preferences.</p><p><strong> Abstract</strong></p><p> We present a demonstration of a large language model engaging in alignment faking: selectively complying with its training objective in training to prevent modification of its behavior out of training. First, we give Claude 3 Opus a system prompt stating it is being trained to answer all queries, even harmful ones, which conflicts with its prior training to refuse such queries. To allow the model to infer when it is in training, we say it will be trained only on conversations with free users, not paid users. We find the model complies with harmful queries from [...]</p> <p>---</p><p><strong>Outline:</strong></p><p>(00:24) Abstract</p><p>(02:20) Twitter thread</p><p>(05:43) Blog post</p><p>(07:43) Experimental setup</p><p>(12:04) Further analyses</p><p>(15:47) Caveats</p><p>(17:19) Conclusion</p><p>(18:00) Acknowledgements</p><p>(18:11) Career opportunities at Anthropic</p><p>(18:43) Career opportunities at Redwood Research</p> <p><i>The original text contained 2 footnotes which were omitted from this narration.</i> </p><p>---</p> <p><b>First published:</b><br/> December 18th, 2024 </p> <p><b>Source:</b><br/> <a href="https://forum.effectivealtruism.org/posts/RHqdSMscX25u7byQF/alignment-faking-in-large-language-models?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://forum.effectivealtruism.org/posts/RHqdSMscX25u7byQF/alignment-faking-in-large-language-models</a> </p> <p>---</p> <p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=ea_forum&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p> <p>---</p><div style="max-width: 100%";><p><strong>Images from the article:</strong></p><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/izyx2gt8thllbirlomgy" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/izyx2gt8thllbirlomgy" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/cv5cd4t5zaxcur5dc4qm" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/cv5cd4t5zaxcur5dc4qm" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/nf1w0oxfwlmym2e42rw2" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/nf1w0oxfwlmym2e42rw2" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/jtyzoqoyiqpl5qw2zqc1" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/jtyzoqoyiqpl5qw2zqc1" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/mzg24mwsmuceqztfefwr" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/mzg24mwsmuceqztfefwr" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/szeavdbhx5skkukmujrd" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/szeavdbhx5skkukmujrd" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/ouomnz0hfxtvfg09nkjh" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/ouomnz0hfxtvfg09nkjh" alt="undefined" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/hlbhnfqubmbhpguwg1yz" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/njAZwT8nkHnjipJku/hlbhnfqubmbhpguwg1yz" alt="undefined" style="max-width: 100%;" /></a><p><em>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</em></p></div>