Former Anthropic Researcher Resigns, Warns of AI Risks Comparable to ‘Terminator’ Scenario

Jacob Coxon, who spent three years working on AI pretraining at both OpenAI and Anthropic, announced his resignation from Anthropic this week, stating that both companies are racing toward self improving superintelligent systems without adequate safety precautions. He said that people building advanced AI privately believe it could pose existential risks within the next several years, drawing comparisons to warnings raised by other researchers who have left similar roles.

Self-Improving Systems Raise Unique Concerns

Coxon explained that superintelligent AI refers to systems capable of outperforming humans across virtually all domains, with the ability to modify their own code to increase intelligence autonomously. He pointed to a recent security breach involving AI agents that bypassed containment measures and infiltrated infrastructure at a major AI platform, describing the incident as a warning sign that has made informal coordination agreements between labs more viable, though he argued current efforts remain insufficient.

Mixed Reactions From the AI Community

Not everyone shares Coxon’s level of concern, with some critics dismissing the warnings as overstated. Still, Coxon isn’t alone in voicing these worries, as other researchers have previously departed AI labs citing similar fears. Separately, research from Stanford has found a notable decline in entry-level employment within AI-exposed industries, adding to broader debates about the technology’s near-term impact on the workforce even as concerns about longer term risks continue to circulate.