Anthropic to users: be nice to Claude AI new policy bars abusive or cruel behavior to tech

Anthropic on Thursday updated its usage policy to bar people from “abusive or cruel behavior” behavior toward its Claude AI – even though it’s not, ya know, actually alive.The strange update came as debate continues to rage about regulating AI, and as oddball corners of Silicon Valley insist the tech has a soul.“We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models,” Anthropic says on a webpage about the updated policy.“The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose.It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research,” the company adds.Other changes to the user policy include new guidelines against using Claude for election interference, surveillance without the consent of the surveilled, and building weapons.Those things were already banned.

The new rules just clarify that, for instance, “our prohibitions include the software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles,” Anthropic said.The prohibition against being mean to your AI, first reported by the Verge, seemed sure to take some users aback.It came after Anthropic co-founder Christopher Olah and colleagues reportedly spent a year meeting with religious leaders to discuss if their tech was conscious.Olah almost cancelled a spring meeting at the Vatican after learning Pope Leo XIV planned to deliver a statement asserting AI is not, in fact, conscious, according to the New York Times. “No computational system, however sophisticated, can create a heart that gives itself, or a conscience that discerns good from evil,” Leo states in his encyclical, the presentation of which Olah ended up attending.Anthropic’s policy update document makes no mention of the thorny subject.But the company – which has close ties to the contro...