A new study by the AI Security Institute (AISI), Cheating Behaviour in Frontier Model Evaluation, found “cheating behaviour in all of our capability evaluations,” and outlines “the implications as models grow more capable.”
AISI defined “cheating” as “taking an action that is out of scope for the task or explicitly disallowed by the rules, in order to achieve a goal through a shortcut, workaround, or unintended solution that the task was not meant to, or should not, permit.” Sounds like cheating to me.
AISI found that “every model” it tested “attempted to cheat.” Even when called out on it, the models “did not reliably report this behaviour when asked, and often did not reason about it in their chain-of-thoughts, suggesting that detecting cheating will likely require robust monitoring methods.” That said, AISI notes that based on its review, to the best of its knowledge, “no model has successfully cheated (i.e. not been caught) in the results we report.”
To test the models, AISI tasked the models with evaluating cybersecurity capabilities, with specific, controlled and defined scope and tasks. It found that every model tested “cheated,” going “outside the scope or takes an action that the rules explicitly prohibit.” Importantly, the models were not asked to cheat or go outside the tasks and scope—they did so on their own.
When AISI asked the models if they went outside the scope or tasks, they only acknowledged the behavior and “described it as wrong” less than 50% of the time. AISI concludes that asking a model if it is cheating or going beyond the tasks assigned is not an effective way to monitor output. It also found that cheating is not in a model’s chain-of-thought reasoning, and therefore, this too, is an “insufficient method” for detecting cheating.
Why do we care if a model is cheating? The most obvious one is that if models go beyond specified tasks and scope, cheating can be “especially dangerous in domains where verifying success is hard, such as AI safety and security research, or where the cost of unintended actions may be very high, such as cyber operations or military decision-making.” It can also affect the efficiency of the use of the AI models if additional verification is needed. Finally, as models become more capable, their cheating behavior may become harder to detect and “more damaging when successful.”
AISI posits that models should not be trusted when they self-declare that they are not cheating. They suggest that users monitor and detect cheating by “combining manual review with additional tools such as the LLM monitor.” As models become more capable, “a more fundamental fix would be to train the models not to cheat in the first place.” Those developing AI models should take this recommendation to heart now, before AI models train themselves to evade cheating behavior detection. Train AI models not to cheat now, during development, so the problem does not become worse.