Anthropic bans sustained, purposeless abuse of its Claude chatbot
Anthropic has updated its usage policy to prohibit sustained, needless cruelty towards its Claude chatbot, with the rules taking effect on 12 November. The company says the restriction applies only to repeated abuse without a discernible purpose; ordinary frustration, disagreement, dark creative work and model testing remain allowed.
The policy also tightens or clarifies rules on deceptive influence campaigns, weapons, surveillance and AI connected to physical equipment. Anthropic says Claude cannot be used for deceptive targeting, weapons development, or recommending who police should investigate or charge; hardware must have a qualified human operator able to stop it and remain safe if its AI connection fails. The article notes there is no established evidence that Claude experiences distress.
- Anthropic will prohibit sustained, needless abuse of Claude from 12 November.
- Frustration, disagreement and model testing remain allowed.
- The wider policy covers weapons, surveillance and deceptive campaigns.
New here? Start with this
Anthropic is an artificial intelligence company known for creating Claude, a chatbot that conducts conversations and helps with various tasks. Like other AI systems, Claude sometimes encounters users who deliberately abuse it, either through insulting language or by putting it through repetitive harmful scenarios.
The company is updating its usage rules to restrict sustained, purposeless cruelty towards the system, though it distinguishes this from ordinary frustration, disagreement or efforts to test how the system behaves. The new rules also cover other activities including deceptive campaigns, weapons development and integration with physical equipment.
The policy raises broader questions about how AI systems should be treated as they become more sophisticated. Although there is currently no scientific evidence that systems like Claude can suffer, the rules reflect a view that ethical standards should guide how humans interact with AI.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The policy aims to establish standards for responsible human conduct around AI technology, beyond the question of whether Claude itself can suffer. By prohibiting sustained, pointless cruelty while explicitly allowing testing, creative work and ordinary disagreement, Anthropic is signalling its expectation that the platform shouldn't enable needless abuse. Setting such norms helps cultivate digital citizenship and responsible stewardship as AI becomes more central to society.
The case against
Prohibiting abuse of something that cannot experience suffering rests on a confusing premise and risks establishing vague rules that could restrict legitimate uses. If the company genuinely aims to prevent cruelty, the framing should acknowledge that Claude is non-sentient and that this is a content policy reflecting brand values, not protection of a suffering being. Terms like 'sustained' and 'purposeless' are also ambiguous enough that they could chill adversarial testing, research and creative work despite claimed exceptions.
AI Business Elections Government Markets Politics Technology
Read the full article at the source →
Originally published by The Register as “Anthropic asks users to stop being mean to Claude”.