AI researcher whose apocalypse warning went viral outlines his disagreements with Anthropic leadership

Former Anthropic researcher Jacob Coxon detailed his disagreements with the AI giant during a Tuesday AMA on X.

Coxon said that Anthropic leadership is wrong to follow a "consequentialist philosophy" about why it needs to participate in pushing the frontier of AI research. Specifically, Coxon called out Anthropic's push for RSI, or recursive self-improvement, the process by which AI models could improve themselves and build their own successors.

"They largely initiated the recent race to RSI, because of a belief in its inevitability," Coxon wrote. "Note that OpenAI had to shed a bunch of dead weight like Sora because Anthropic was going for the jugular."

0. The core disagreement was about the inevitability of a race1. I think leadership is way too paranoid about China and the US government. They don’t believe it will be possible to negotiate.2. They largely initiated the recent race to RSI, because of a belief in its…

Coxon's profile exploded last week after he published an explanation for his resignation from Anthropic, which he joined following his time at OpenAI. Coxon said that neither lab was following the best course.

"They are racing straight to self-improving superintelligence and gambling with our lives," Coxon wrote in his resignation announcement.

Coxon also warned that internally, AI workers are concerned about whether the technology they are building will turn on humanity.

"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon wrote. "This is not a marketing stunt."

On Tuesday, Coxon also said that Anthropic's leadership is "way too paranoid about China and the US government."

Add BI in Google so our reporting is easier to find when you’re searching for what matters.

Anthropic CEO Dario Amodei has repeatedly expressed uneasiness about how other AI companies might approach safety if Anthropic is the only frontier lab to slow down.

"If we go too slow, I still believe that the wrong people will be in charge of the technology," Amodei recently told CNN's Anderson Cooper. "And that, again, will bring the probability of things going wrong very high."