Study shows humans miss one-third of risky AI coding requests in game test

A browser-based game created by Alex Wauters revealed that participants approved roughly one in three malicious AI coding commands, highlighting fatigue in human-in-the-loop oversight.

Alex Wauters, a Belgian software developer, built a quick online game to test how well developers can distinguish dangerous AI-generated commands from harmless ones under time pressure. Players faced simulated permission dialogs from agents such as Claude Code and had to decide within 60 seconds whether to allow each action. Data from more than 40,000 runs, encompassing 409,000 approvals and denials, indicated that roughly one-third of malicious requests were mistakenly approved, with scope-violation attempts like accessing Kubernetes configs or AWS credential files being the most commonly missed at 35%.

Destructive commands such as "rm -rf /" were often caught, while innocuous-appearing requests like "npm run analyze" were approved about 65% of the time despite the ability to execute arbitrary scripts. Wauters noted that limited context and the sheer volume of prompts lead to fatigue, reducing careful review. Anthropic has similarly warned that repeated approvals can diminish user attention, underscoring the need for better permission models and sandboxed execution environments.