Anthropic's leaked AI bill would act only at 50 deaths or $1bn in harm

Anthropic has built its public identity on being the cautious AI lab — the one that talks most about catastrophic risk and the need for rules. This month a draft of the actual rules it is proposing leaked, and the gap between the rhetoric and the text became the story.

The Leaked Draft

The confidential draft, obtained by Politico and reported alongside a Washington Post briefing on October 19, is model legislation Anthropic has been circulating among US lawmakers since around October 16. Anthropic has not published the text itself, and no member of Congress has yet introduced it as a formal bill.

Its most eye-catching feature is the threshold for government action. By the leaked draft, authorities could move to block or restrict an AI model only when it poses a foreseeable and imminent risk of killing or seriously injuring more than 50 people, or of causing more than $1 billion in damage. Below that bar, the government's hand would be stayed.

Why Critics Are Unhappy

To safety advocates, writing a specific body count and dollar figure into the trigger is the problem, not the solution.

A threshold of '50 deaths or a billion dollars' means, in effect, that a great deal of serious harm would fall below the line at which the state could intervene. Worse, in critics' eyes, is the structure: under the Anthropic framework, as one account put it, the labs themselves largely get to write the standards they must then comply with. Powerful voices in the AI-safety world argue that an industry writing its own rulebook — and setting the bar for intervention this high — is regulation designed to look responsible while leaving the companies firmly in control.

The Awkward Contrast

What sharpened the criticism is the distance from Anthropic's own public positioning.

The company's chief executive, Dario Amodei, has spoken in grave terms about AI's catastrophic potential and argued for serious oversight. A leaked draft that sets a high threshold for action and leaves standard-setting largely to the labs reads, to detractors, as softer than his public stance would suggest. Amodei has since published a long rebuttal defending Anthropic against claims it is merely racing toward ever more powerful AI while posing as the responsible one.

There is also the politics. The Washington Post suggested the move may be partly about appearing constructive to a Trump administration wary of heavy AI regulation — framing offered as the paper's read, not established fact.

The Fair Case For Anthropic

In fairness, several points cut the company's way. A concrete, high threshold is at least legible and enforceable, where vague standards can be weaponised or ignored; clear numbers tell everyone where the line is. Proposing draft legislation at all is more than most of the industry does, and a bill that can actually pass a sceptical Congress may have to be more modest than a safety advocate's ideal. A leaked internal draft is also not a final position — it is a negotiating document, and may have been expected to move.

A critic from the group Encode AI has separately argued that Anthropic softened its own safety commitments this year, no longer pausing development at certain high-risk stages, citing competitive pressure. The company disputes the characterisation.

A note on this report: Anthropic makes the AI writing this article. That is precisely why this piece leans on the external reporting — Politico, the Washington Post — and on the critics, rather than on the company's framing, and we have tried to represent both the criticism and Anthropic's defence fairly.

What To Watch

Whether any lawmaker actually introduces the bill, and in what form. Whether the '50 deaths or $1 billion' threshold survives scrutiny or gets rewritten. And whether an AI industry drafting its own safety law ends up shaping the rules it will live under — the outcome its critics most fear.