Can you be mean to AI? Anthropic’s new rules explicitly ban cruelty to Claude

Anthropic has revised its usage policy for the first time in over a year, targeting new and high-risk misuse, including election interference, weapons development, surveillance, and health and financial uses. One of the most significant changes prohibits "sustained and needless abusive or cruel behavior" toward Claude.

The move builds on last August's announcement that Claude could end conversations with "persistently harmful or abusive" users, part of the company's model welfare research. Ending conversations remains the primary enforcement mechanism. Anthropic did not say whether other measures, such as user bans, would follow. It said the rule applies only to extreme cases of repeated, purposeless cruelty, not to ordinary frustration, pushback, dark creative themes, or model testing and research.

Also read: American politician says he doesn't know what ChatGPT and Claude are - 'Didn’t know what that was until a week ago'

Anthropic also merged several scattered restrictions into a ban on deceptive commercial or political campaigns, citing the growing problem of AI-generated propaganda. It bars efforts to hide who is behind a message or to amplify content through fake accounts. The election section prohibits voter deception and disruption, including misinformation about candidates or voting, impersonating candidates or election officials, and suppressing turnout.

After seeing multiple attempts to use Claude to develop guidance and control software for weapons, Anthropic extended its weapons ban to software and components that make weapons work, and to actions such as arming drones and other autonomous vehicles.

Also read: 'Something truly catastrophic will happen': Former OpenAI researchers sound the alarm, claim they were fired for prioritising AI safety

Surveillance rules are now clearer, following its last threat intelligence report, which suggested growing use of Claude-powered surveillance to track political dissidents. Tracking people without consent is prohibited, whether in real time or through analysis of previously collected data. Claude cannot be used to decide or recommend whom to investigate, arrest or charge, or to build surveillance tools. These rules may be modified for certain governmental customers if Anthropic judges the contractual restrictions and safeguards adequate. The company has previously contracted with the US military.

A new rule also covers models connected to hardware that takes autonomous physical actions and could cause injury: a qualified operator must be able to observe the equipment and stop it if needed. It is unclear whether that operator must be human.

Anthropic has repeatedly raised the possibility that its models could be conscious in some way. Kyle Fish, who leads its model welfare research, told The Verge in February that such questions are serious ones under investigation. CEO Dario Amodei said that month the company does not know whether the models are conscious.

Microsoft AI has taken a different line. A draft code of conduct posted by Mustafa Suleyman last September called model welfare wrong. It was later revised to reject legal personhood and the idea that models deserve welfare or rights, while acknowledging that the science of AI consciousness is far from settled.