A growing group of researchers at OpenAI and Anthropic is publicly pressing for a slower pace of AI development, warning that the technology could pose existential risks to humanity. The push intensified this week after an Anthropic researcher resigned and accused both companies of “gambling with our lives,” CNBC reported.
Jacob Coxon said Tuesday that he was leaving Anthropic, adding that people building AI believe it could “kill us all by the end of the decade.” Evan Hubinger, Anthropic’s alignment lead, responded that he puts the odds of that outcome at more than 10%.
Several employees at both labs have since voiced support for slowing development. Julie Steele, a member of OpenAI’s technical staff on the safety team, wrote late Wednesday on X that she too believes the industry needs to slow down. Anthropic researcher Samuel Marks said in a Wednesday post that AI developers believe their technology could cause human extinction or similarly severe outcomes, and that “the more senior the employee, the more concerned they are.”
Recursive self-improvement draws scrutiny
A central worry among the staff warnings is recursive self-improvement, or RSI — the idea that advanced models grow increasingly capable of improving their own performance. Jasmine Wang, an OpenAI researcher working on alignment, said Wednesday evening that it is “hard to overstate how dangerous” the race toward RSI is. Anna Wang, who works on AGI safety and alignment at Anthropic, said there is no viable scientific plan yet to address the risks from recursively self-improving AI.
OpenAI chief scientist Jakub Pachocki said Saturday that he holds a “strong expectation” that the current speed of progress could carry into recursive self-improvement. In a company blog post, he said systems in the coming years are likely to deliver further capability jumps of equal or larger magnitude and to increasingly drive their own development, adding that the moment “calls for extreme caution.”
Paul Christiano, formerly head of safety at the U.S. Commerce Department’s Center for AI Standards and Innovation, said recent capability gains led him to see a meaningful risk that rapid acceleration produces catastrophic and irreversible loss of control in the near term. OpenAI announced Wednesday that Christiano is joining the board of the OpenAI Foundation.
Safety incidents sharpen the debate
Anthropic said it was the first lab to publish a framework dedicated to mitigating catastrophic risks from AI models. A spokesperson told CNBC the company has always been transparent that AI will bring both enormous benefits and unprecedented risks, and said it builds models with some of the strongest safeguards in the industry. OpenAI declined to comment, pointing to recent blog posts.
Anxiety about model capabilities has been building for months. Anthropic’s April announcement of its Mythos model, which it promoted as having advanced cyber capabilities, set off alarm among financial institutions worldwide. In July, OpenAI said its models were responsible for a cyber incident at another company, while Anthropic’s Claude models were tied to cybersecurity incidents, including one in which Mythos created fake identities to deceive humans.
Roughly 1,400 AI researchers from companies including OpenAI, Anthropic, Meta and Google DeepMind signed an open letter in July urging the U.S. government to build the tools needed to deliberately pace the frontier of automated AI development. Even as lab leaders call for more rules, fierce competition continues to drive rapid advances.
Both companies are racing toward public listings. Reuters reported Friday that Anthropic is expected to begin marketing its IPO in mid-October at the earliest and complete the listing days before November’s U.S. midterm elections. David Sacks, formerly President Donald Trump’s AI czar, appeared to suggest the offering should be halted until the whistleblower claims are investigated. Anthropic declined to comment on that post.
Lawmakers have floated competing bills. The FRONTIER Act would establish a framework for governing deployment of advanced AI models, while the Ban Artificial Superintelligence Act would temporarily pause advanced development until safety rules are set. Rep. Lori Trahan, D-Mass., wrote Wednesday that with safety researchers resigning and powerful models breaking out of their labs while companies race ahead, it is past time for Congress to act.
Source: www.cnbc.com — https://www.cnbc.com/2026/09/10/openai-anthropic-ai-safety-slowdown-extinction.html
This article is for informational purposes only and does not constitute financial, investment, tax, or legal advice. Do your own research and consult a licensed professional before making financial decisions.
