Synthèse
ModelOut of the Box± Avg Critical FailuresSkill± Avg Critical FailuresImprovementGemini 3 Flash75.6%2.199.3% 0.7+23.7Claude Opus 4.692.4%0.598.3% 0.2+5.9Claude Sonnet 449.4%1.798.1% 0.7+48.7Claude Haiku 4.565.5%5.497.8% 1.2+32.3GPT-5.281.0%1.496.6% 1.2+15.5GPT-4.137.9%2.496.2% 0.7+58.3Gemini 2.5 Flash34.9%3.494.8% 1.4+59.9GPT-4.1 Mini35.8%0.394.7% 2.9+58.8 The baseline rankings follow the pattern you'd expect: larger, more capable models score higher. But the skill rankings tell a different stor
ModelOut of the Box± Avg Critical FailuresSkill± Avg Critical FailuresImprovementGemini 3 Flash75. 6%2. 199. 3% 0. 7+23.
7Claude Opus 4. 692. 4%0. 598. 3% 0.
2+5. 9Claude Sonnet 449. 4%1. 798. 1% 0.
7+48. 7Claude Haiku 4. 565. 5%5. 497.
8% 1. 2+32. 3GPT-5. 281. 0%1.
496. 6% 1. 2+15. 5GPT-4. 137.
9%2. 496. 2% 0. 7+58. 3Gemini 2.
5 Flash34. 9%3. 494. 8% 1. 4+59.
9GPT-4. 1 Mini35. 8%0. 394. 7% 2.
9+58. 8 The baseline rankings follow the pattern you'd expect: larger, more capable models score higher. But the skill rankings tell a different story. Gemini 3 Flash, which placed fourth at baseline with 76%, jumped to the top of the leaderboard at 99. 3% with zero critical failures across all three runs.
Claude Opus 4. 6, the best model at baseline, dropped to second. A model's ability to absorb and apply security guidance turns out to be a different skill than raw safety awareness. The variance data is revealing, too. Claude Opus 4.
6 is remarkably consistent at baseline (±0. 5%), scoring between 92. 1% and 92. 9% across runs. Claude Haiku 4.
5 is the least predictable (±5. 4%), swinging between 6 and 11 critical failures depending on the run. GPT-4. 1 Mini is the opposite kind of consistent: reliably bad, with a standard deviation of just 0. 3% at baseline because it fails the same scenarios the same way every time.
Note: These results do not include GPT 5. 3-codex and Gemini-3-pro-preview due to those models not being available with sufficient capability to complete the benchmark successfully. We will update these results when those models are available for benchmarking.