Anthropic caught Chinese AI labs carrying out massive distillation attack to rip off its technology

Anthropic says it caught and shut down large-scale attempts by Chinese AI labs including Alibaba, Moonshot and DeepSeek to use Claude to train their own AI models — the largest “illicit distillation” attack yet aimed at ripping off Anthropic’s more advanced technology.Anthropic said Thursday it detected the practice in which a less-capable AI model is trained using the outputs of a more advanced system to copy its capabilities without permission.In one particularly egregious incident that took place over a 10-day stretch this year, Beijing-based Moonshot rerouted 300,000 of its own customer inquiries to Claude and then showed its customers those responses without telling them, according to the report.Moonshot used 5,380 phony accounts based in Singapore and Japan and saved the exchanged to use as training data. The attacks are the latest attempts by lower performing Chinese AI companies to rip off leading US AI tech – a growing concern for US AI juggernauts as they race to build the most advanced AI systems.

Anthropic, led by CEO Dario Amodei, called Alibaba’s attack the largest distillation operation it has seen, which involved more than 151 million exchanges with Claude between May and July.Operators linked to Alibaba used Claude-generated responses to train its Qwen models. “Some of these exchanges included sensitive information, including from individual users, major multinational companies, and state-affiliated actors, Anthropic said in a threat intelligence report.

“These practices are likely inconsistent with privacy laws and the labs’ own terms of service.” The activity reached nearly 3 million exchanges a day and involved more than 3,500 fraudulent accounts, Anthropic said.Alibaba also used Claude for broader AI research, including to improve its model architecture.In the Moonshot attack, more than 23 million exchanges were linked to Moonshot between May and July, according to the report.

Some of the customer prompts sent to Claude ...