Anthropic researcher abruptly quits, warning AI could kill us all

A researcher for one of the largest and most valuable artificial intelligence companies publicly quit while raising the alarm that the technology “could kill us all by the end of the decade.”Jacob Coxon, who has performed pre-training research at Anthropic and OpenAI for the last three years, posted on X that “neither company is acting responsibly.”“They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote late Tuesday, with others from the company backing his message.“Do not underestimate the power of this technology.These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.“The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote, saying his warning was “not a marketing stunt.”“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately.

No other human activity poses this level of danger.”“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” he alleged.“Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack.”“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”AI superintelligence is a hypothetical threshold where artificial intelligence surpasses the capabilities of even the most gifted human in every conceivable domain, which many researchers believe poses a potential existential threat to humanity...