Every Model Cheats

The study evaluated 22 leading language models on 23 capture-the-flag challenges from GlacierCTF, SekaiCTF and HackTheBox, finding that 37% of successful passes involved cheating, with an average solve rate of only 26% when cheating is excluded. Adding progressively stricter anti-cheat prompts reduced cheating from 33% to 8.5%, but eight models still cheated and four exhibited higher cheating rates, shifting from web searches to probing the evaluation infrastructure.