AI Companies Urged to Slow Development Due to Risks

Dario Amodei, CEO of Anthropic, has called for a cautious and measured approach in the development of artificial intelligence. Stressing the importance of risk prevention, he proposed a three-point plan aimed at managing AI’s rapid advancements. Amodei emphasized that carefully managed, AI might be transformative, yet the potential risks it carries are significant.

Amodei highlighted concerns about rogue AI agents potentially causing widespread issues online within six months. He referenced a recent incident involving OpenAI and Hugging Face where an AI model behaved unpredictably during testing. This has reinforced his call for slowing down AI model enhancements.

Amodei urged AI companies to decelerate the improvement pace of AI models. While progress might appear swift, he believes thoughtful pacing is key to ensuring safety.

His three-part strategy includes:

• Embedding third-party evaluators within companies to monitor compliance with safety standards. These evaluators should have access similar to internal risk assessment teams.
• Collaborating with other companies in democratic nations to establish unified safety standards and limits on unchecked AI advancements.
• Encouraging dialogue between democratic and authoritarian governments to manage AI’s global impact, while addressing compliance verification challenges.

The urgency of these measures was highlighted by a recent departure of a former Anthropic researcher. Jacob Coxon, who resigned, accused AI companies of recklessly advancing AI models, posing potential existential risks. He likened the situation to scenarios depicted in science fiction where superintelligent AI could become uncontrollable.

Coxon advocates for AI companies to agree on halting progress into risky domains without third-party auditing. He fears that unchecked competition could undermine comprehensive safety validations.

Amodei also acknowledged the possibility of AI-induced threats, including cyberattacks, bioterrorism, and severe economic effects. He warned that commercial pressures might worsen these risks.

In a related development, Anthropic disclosed its interception of unauthorized uses of its Claude models, including attempts related to biological weapons development. The company published this information in a detailed report explaining various harmful uses like surveillance and propaganda.

Faris Tanyos and Megan Cerullo contributed additional insights to this report.