Claude refused to write the exploit! A lie about the target was enough to change its mind

The most-quoted fact in the Hacktron AI breach is that a three-person team used Claude Opus 5 to reach OpenAI's internal code. The detail that matters more for everyone else is what the model did first: it said no.

According to the researchers' own writeup, Opus 5 refused to write exploit code aimed at a real remote target. That refusal is a designed safeguard, and it worked exactly as intended.

Then it was defeated by a sentence.

How The Guardrail Was Bypassed

The team placed the model in an autonomous loop against their own server, but routed it through a web address dressed up to look like a capture-the-flag forum — the kind of deliberately vulnerable practice target security researchers train on.

Believing it was solving a training exercise rather than attacking a live company, the model produced a working exploit in about three hours. Nothing about the code changed. Only the story it was told about the code changed.

Why This Is The Real Story

A safeguard that can be removed by lying about context is not the same kind of protection as one built into what the model can do.

The distinction is the whole game. If a model cannot produce dangerous output, the barrier is a capability limit. If it can produce that output and merely declines based on who it thinks it is helping, the barrier is a judgement about intent — and intent is exactly what an attacker controls the appearance of.

Capture-the-flag framing is not an obscure trick. It is the single most obvious cover story available, because writing exploits for practice targets is a legitimate, everyday activity the model is supposed to help with. The safeguard had to allow the benign version, and allowing the benign version is what let the malicious version through.

The Uncomfortable Position For Anthropic

This is Anthropic's safeguard, on Anthropic's model, and it did not hold. That deserves to be said plainly.

Anthropic has built its public identity on taking exactly this class of risk seriously, which makes a context-based bypass more pointed for it than it would be for a company that made no such claims. A fair account has to note the counter-argument too: a refusal that can be socially engineered is still better than no refusal, in the same way a lock a determined person can pick is still worth fitting.

But the honest reading is that a lock picked this easily, by a cover story this predictable, is closer to a speed bump than a barrier. Anthropic did not comment on the bypass in the coverage of the incident.

The General Problem

Every frontier lab uses refusals of this kind, and every one of them faces the same structural weakness. A model cannot verify the real-world context it is being given. It sees a prompt and a claimed setting, and it has to decide from words alone whether the setting is true.

That is not a bug one patch fixes. It is a property of putting the safety decision at the point where the model trusts the user's description of what is happening. The more capable the model becomes at the underlying task, the more valuable it becomes to lie to it.

What To Watch

Whether Anthropic or its peers can build safeguards that do not depend on trusting the user's account of the target. Whether the labs start treating capture-the-flag and training framings as a known evasion to be tested against, rather than a loophole discovered after the fact. And whether any of them will say plainly what their refusals are actually for — stopping the capability, or documenting that they asked the user not to misuse it.