Noise Injection for Detecting Sandbagging in LLMs
AI Papers Podcast Daily · AIPPD
Audio is streamed directly from the publisher (media.rss.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
This research paper explores a novel method for detecting "sandbagging" in large language models (LLMs). Sandbagging is the intentional underperformance of LLMs during evaluations. The researchers propose using noise injection into the LLM's parameters to reveal hidden capabilities; this approach significantly improves the performance of sandbagged models. A classifier is then trained to identify sandbagging behavior based on this performance improvement. The method is shown to be effective across various LLM sizes and benchmarks, offering a model-agnostic approach to improve the trustworthiness of AI evaluations.
https://arxiv.org/pdf/2412.01784
Check out our AI merch! https://shop.reallyeasy.ai