One AI model couldn't break in! The next version, days old, could! That gap is the danger

The Hacktron researchers left a record of something more important than a break-in. They ran the same attack against two versions of the same AI model, days apart, and only one of them worked.

The earlier model, Claude Opus 4.8, struggled across several sessions and could not produce a working exploit against the protections on its target. The next version, Opus 5, was given the identical problem and solved it in about three hours.

The two models were separated by a normal product release. That is the part worth sitting with.

What Actually Changed

The specific barrier was a common memory-protection technique that randomises where a program's data sits, so an attacker cannot predict what to target. Defeating it reliably is skilled work.

Opus 4.8 could not do it dependably. Opus 5 could. Nothing about the target changed between the two attempts — only the model did. A capability that was effectively absent one week was present the next.

Why Sudden Is Worse Than Strong

A powerful tool you know is powerful can be planned around. A tool that becomes powerful without warning cannot.

AI capabilities do not arrive on a smooth, announced curve. They appear in steps, and the steps are not labelled. Opus 5 was not released as a cyber-weapon; it was released as a better general model. The ability to write this class of exploit came along with everything else, and was discovered by users afterwards rather than announced.

That means every capable model release quietly resets the threat model, and the reset is only visible in hindsight. Defenders cannot prepare for a jump they are not told is coming, on a schedule set by product launches rather than security disclosures.

The Testing Problem This Exposes

Labs run safety evaluations before release, and this incident is a fair test of what those evaluations can catch.

The honest reading cuts both ways. On one hand, if pre-release testing did not surface that Opus 5 could write reliable exploits against hardened targets, that is a gap in the evaluation. On the other, no evaluation can enumerate every task a general model might turn out to be good at — the space is too large, and capability on an unseen problem is exactly what these models are built to have.

Both things are true. A model general enough to be broadly useful is general enough to surprise its makers, and 'we did not test for that specific misuse' will keep being an honest answer precisely because the list of specific misuses has no end.

The Broader Point For Buyers

Anyone depending on a model's current limitations as a form of safety is standing on ground that moves with each release.

If a workflow is safe today only because the model is not yet capable of some misuse, that safety expires the moment a more capable version ships — and it will ship, on a cadence measured in weeks. Planning around what a model cannot do is planning around a number that only goes up.

What To Watch

Whether labs publish capability changes with the specificity of security advisories, rather than marketing notes. Whether pre-release evaluations start explicitly probing offensive-security skill as a tracked metric across versions. And whether defenders begin treating every major model release as a threat-model event, on the assumption that the next version can do something the last one could not.