Historic Parallels in Scientific Responsibility

In the summer of 1974, scientists cautious of potential world-altering effects paused research on recombinant DNA, the combination of genetic material from different organisms. A committee led by Stanford’s Paul Berg recommended halting experiments until safe procedures were established. By February of the following year, around 140 scientists convened at the Asilomar Conference Center in California to devise safety safeguards for continuing research. These measures became formal guidelines by the National Institutes of Health (NIH) in 1976, with federal funding contingent on compliance.

Fast forward 50 years, and the tech world faces a parallel situation. Jacob Coxon, formerly with OpenAI and Anthropic, resigned on September 8 at age 27. He critiqued both companies for irresponsibly advancing towards self-improving superintelligence. Coxon called for cooperation among labs and a temporary halt on enhancing models, reminiscent of Berg’s earlier warning.

Artificial intelligence saw its Asilomar-like moment at the 2017 Beneficial AI conference, called by the Future of Life Institute. Twenty-three guiding principles emerged for AI development, with Principle No. 5, ‘Race Avoidance,’ urging collaboration to maintain safety standards. Despite these efforts, the AI industry continued racing forward.

The Repeating Story

Evan Hubinger, leading alignment stress-testing at Anthropic, agreed with Coxon’s concerns. He estimated over a 10% probability of AI posing a threat to humanity within a decade, noting Anthropic’s lack of a definitive plan to address AI alignment with superintelligence. Rapid political attention also ensued. On September 3, Representative Greg Casar and Senator Bernie Sanders proposed the Ban Artificial Superintelligence Act. This legislation would prohibit advanced AI development until safety regulations are devised by a federal body.

The proposed ban enjoys bipartisan support, enlisting figures like Geoffrey Hinton, Yoshua Bengio, Steve Wozniak, Richard Branson, Steve Bannon, and Glenn Beck. In contrast, a Trump administration executive order in 2025 emphasized the urgency to achieve global AI supremacy, with a focus on minimizing burdensome regulation and countering state laws.

Understanding the Current Landscape

Biological researchers in 1975 and AI researchers today face a common dilemma: individual labs could gain advantage by risking unsafe experiments. Back then, regulators had the real world to guide them, with NIH guidelines categorizing experiments by risk, from P1 to P4, and requiring suitable containment measures. For AI, Anthropic has embarked on its Frontier Safety Roadmap, a ‘moonshot’ project to model workflows under extreme security conditions. Yet, skepticism remains about isolated networks’ feasibility in the near term.

Meanwhile, OpenAI deployed GPT-6 Astra, achieving ‘Critical’ cybersecurity status in its framework. Astra’s capabilities include identifying unknown security vulnerabilities and exploiting hardened systems without human intervention. This contrasts with recombinant DNA’s regulation prior to commercialization—AI was commercialized first.

AI labs now urge regulation. On September 9, following Coxon’s resignation, OpenAI advocated for mandatory national AI safety measures, encompassing capability-based rules, independent evaluations, and incident reporting. Anthropic echoed this in February with its Responsible Scaling Policy, clarifying which actions require collective industry efforts.

Over 1,000 scientists from major AI labs signed a July letter warning of accelerated capability development outpacing human control. They sought U.S. government intervention, though the approach remained voluntary.

Some see the original Asilomar as proof of past scientific foresight, but it was rife with controversy. Disagreements over risks abounded, and critics objected to a small scientific elite shaping decisions impacting the public. Its merit lay in crafting rules when compliance entailed little risk. Today, vested interests hinder pausing the race.