Anthropic Researcher Warns of AI Dangers and Urges Regulation

The researcher who recently left Anthropic, due to concerns about AI technology, believes hope exists to prevent potential catastrophic outcomes over the next decade. Jacob Coxon, 27, discussed these concerns with Fox News, emphasizing the importance of regulation. On Wednesday’s “Special Report,” Coxon noted that while companies are not acting responsibly, regulation could mitigate the risk of self-improving superintelligence harming human life. He announced his resignation on X, citing corporate negligence in the pursuit of advanced AI.

Coxon, who spent three years conducting pretraining research at OpenAI and Anthropic, does not see the situation as entirely hopeless. He views the potential to regulate AI as “eminently solvable.” He shared with Bret Baier that there is a significant chance, greater than 10%, that AI poses a threat to humanity. However, he is confident that the country can influence this dynamic and reduce the risk.

“It’s not like we think this is some impossible thing,” Coxon stated. “It’s just we’re racing so fast. We haven’t got time to solve it correctly. We’re not being careful.”

Evan Hubinger, alignment science lead at Anthropic, supported Coxon’s concerns in a post, agreeing with the risk assessments by affirming that AI could indeed pose a substantial threat. Hubinger emphasized the lack of a current solution to alignment for superintelligence, yet acknowledged that current AI models present minimal risk.

Coxon highlights that the real threat lies in recursive self-improvement, where AI continues to enhance its capabilities without human oversight. This risk could become significant soon, potentially within a year. He stressed the urgency for international collaboration to oversee AI development, as the technology has significant global implications.

“This is possibly the most dangerous technology that humanity has ever created,” Coxon stated. He called for global cooperation to avoid an arms race, which he described as “disastrous.” Despite these concerns, he acknowledged the potential benefits of AI, urging a cautious approach to its development.

During the Fox News interview, Coxon expressed belief that the challenges posed by AI are solvable if addressed with intention rather than passive acceptance of a potentially negative outcome. He urged for regulatory efforts to maintain control over AI’s progression. Anthropic has yet to comment on these remarks.