06/08/2026
Anthropic, the company behind the Claude AI chatbot, is calling for a coordinated global pause on AI development before humans lose control of it entirely.
Their concern is something called "recursive self-improvement," which is the point where AI models start autonomously designing and upgrading their own successors faster than anyone can put safety guardrails in place (what we've been warning about since 2022). They're urging major AI labs to agree to a verifiable pause so researchers can actually align AI capabilities with human values before things get out of hand.
Anthropic was recently blacklisted by the U.S. government after refusing to let the military use their models for domestic surveillance. So at minimum, these aren't people who just say whatever is convenient for them.
Rival OpenAI pushed back, arguing that governments should set the rules, not private companies.
The University of Toronto recently showed how existing AI tools can already engineer cyberworms that autonomously hijack computer networks. Federal regulations are nowhere near catching up.
My honest take on that debate is several countries are in an arms race for AI dominance. Because of the nature of our system design, there is little incentive to be safe in this process since GDP and dominance are the big incentives. So there is no reason for governments under this structure to regulate AI until someone has won. This is the problem with our existing incentive structures, human and natural wellbeing are not priorities.
