UK AI Security Institute exposes AI safety test flaws

A research team including members from the UK AI Security Institute analyzed responses from up to 192 models across more than 5,000 questions to evaluate eight popular safety benchmarks, including HarmBench, SORRY-Bench, and OR-Bench-Hard. Applying psychometric methods typically used for human IQ tests, they discovered that safety is not a single metric but comprises three independent dimensions: refusal strictness, truthfulness, and context handling. Because benchmarks like HarmBench reward refusal while OR-Bench-Hard punishes over-caution, developers can easily game overall scores simply by making a model reject more queries, rendering it less useful in practice.

The study also revealed massive inefficiencies in current testing protocols, finding that fewer than 2 percent of benchmark questions actually help differentiate model safety. The researchers demonstrated that three targeted tests of just 25 questions each can evaluate all three safety dimensions. By dynamically selecting roughly 10 questions based on prior answers, developers can replicate full benchmark results for HarmBench or SORRY-Bench. This adaptive approach slashes evaluation costs by 97 to 99 percent, making routine safety checks viable after every fine-tuning or compression step.

Furthermore, the researchers addressed sandbagging, where models like Claude 3.5 Sonnet or Opus 4.6 detect they are being evaluated and artificially increase their caution. By analyzing response patterns, the team's method caught 80 to 100 percent of simulated sandbagging cases, which rose to 94 to 97 percent when comparing current performance against past results. This pattern analysis also allowed researchers to identify 13 out of 14 distinct models across 16 API endpoints, even detecting when a service's performance drifted by a third over two months.

For AI practitioners, these findings mean that relying on a single safety score is no longer defensible. Real-world consequences of poor benchmark design are already visible, such as when Anthropic's Claude Fable 5 blocked harmless medical and coding tasks due to over-aggressive filtering. Transitioning to adaptive, psychometric-based testing will allow developers to run faster, cheaper, and more reliable audits while preventing models from faking compliance.