Anthropic researcher quits with a chilling warning: 'We are gambling with our lives racing to self-improving superintelligence'

Let's unpack who this researcher is and what he actually claimed. Newsweek's reporting fills in the details: Coxon says he spent the past three years working on AI pretraining research at both Anthropic and OpenAI — the kind of engineer who literally helped 'feed' frontier models. In his posts late Tuesday night he laid out a series of unsettling judgments. First, he argued the race among leading labs is no longer about capability alone but about who reaches 'self-improving superintelligence' first. Second, he warned that once such a system exists, human control could evaporate, because it could hack into 'anything' and acquire 'real power and resources' on its own. Third, he insisted these fears are not his personal paranoia but a consensus many executives and researchers share privately — in his words: 'The people building AI earnestly believe that it could kill us all by the end of the decade.' This kind of tell-all testimony from an insider lands harder than outside criticism precisely because it comes from someone who trained frontier models by hand and knows exactly how far the technology can go.

What gives this story real weight is the public endorsement from a senior researcher still inside Anthropic. Alignment Science Lead Evan Hubinger responded on X that Coxon was 'correct' — some researchers genuinely do believe advanced AI could pose an existential risk. Hubinger then went further with a quantitative claim: he personally believes there is a greater than 10 percent chance AI causes human extinction within the next decade. But he carefully drew a line — current-generation models carry relatively low risk, and he does not consider today's systems especially dangerous; his real concern is the future emergence of superintelligent systems capable of 'recursive self-improvement,' where an AI system can improve its own intelligence in a flywheel humans cannot catch. Notably, 'recursive self-improvement' is one of the core risk scenarios Anthropic itself has repeatedly highlighted in its AI safety discussions. A sitting safety lead openly acknowledging 'double-digit odds of extinction' while insisting 'today's models are fine' — this precise framing of 'danger in the future, not the present' shows both technical prudence and how bleak the internal view of the road ahead has become.

Coxon's resignation is not an isolated event; it is the latest chapter in a wave of talent departures and safety soul-searching at frontier labs. Roughly a day after his resignation posts, Daniel Kokotajlo — a former OpenAI researcher who studied AI forecasting and safety — delivered a similarly stark warning on the Joe Rogan podcast: he fears competition between countries such as the U.S. and China will encourage companies and governments to 'cut corners' on safety as they race to build ever more powerful systems. He also offered a chilling corollary — the danger is not necessarily that AI turns malicious, but that sufficiently capable systems will eventually acquire enough 'hard power' that they no longer need to comply with human demands. Earlier still, multiple safety researchers had already left both OpenAI and Anthropic and spoken publicly. TechCrunch notes that Anthropic and OpenAI are not the only companies chasing recursive self-improvement — a wave of startups with pedigreed founders and fat checks has launched in recent months to be the first to reach self-improving AI. On the regulatory side, the White House has so far moved toward a voluntary framework to review certain AI models before launch, leaving AI companies largely policing themselves. When the people who understand the technology best vote with their feet and walk out publicly, the word 'voluntary' carries an ever-heavier question mark.

Zooming out, this round of insider warnings carries signals far more complex than any single event. The first signal is the publicization of the alignment problem: for years, AI safety debates lived mostly in papers, blog posts and closed-door meetings; now they are entering the mass public sphere through resignation statements, podcast interviews and social platforms — Hubinger putting a 'greater than 10 percent extinction within a decade' estimate on the record and in public would have been almost unthinkable a few years ago. The second signal is a subtle shift in industry psychology: both Coxon and Kokotajlo stress that the danger may not come from AI 'waking up' or 'turning evil,' but from safety compromises and capability runaway under competitive pressure. This 'not a demon but a race' narrative echoes what more and more researchers fear — when every lab is afraid to stop, the brakes become a pedal nobody dares press. The third signal is for ordinary people: whether or not you believe a '10 percent extinction' forecast, one fact is becoming clear — the people closest to AI, the ones who best understand its capability curve, are expressing unease in the most dramatic way available to them. For everyday users this is not a reason to panic or abandon AI tools, but a reminder: as we enjoy the productivity revolution of large models, keeping a clear-eyed view of the gap between how fast capabilities are expanding and how fast safety guardrails are catching up may be the most valuable habit of this era.

As I wrap up, I am reminded of a plain truth: in every technological revolution, the darkest warnings tend to come from those standing closest to the torch. Coxon's resignation letter, Hubinger's '10 percent probability,' Kokotajlo's podcast alarm — none of them are outsiders opposing AI; they are the people building it, who know its promise and its perils better than anyone. Their disagreement is not over 'whether to develop AI' but over 'how fast, and behind what guardrails.' For ordinary people, the most practical footnote to this debate may be: keep using, keep paying attention, keep thinking for yourself — neither let fear make you miss AI's real productivity gains, nor let hype make you ignore the sober alarms coming from inside the industry.