Anthropic is warning that advanced AI may one day build its own successor, a possibility that could reshape the debate over AI safety, human control and frontier AI oversight.
The San Francisco-based company behind Claude said full recursive self-improvement has not arrived. But it warned that governments and AI companies should prepare for a stage in which AI systems can improve themselves and help design more powerful future models with limited human involvement.
Anthropic Raises Recursive Self-Improvement Risk
Recursive self-improvement refers to a scenario in which an AI system becomes capable of improving its own abilities or helping create a more advanced successor. Anthropic said such a shift could bring major benefits in science, health care and research, but also create serious control risks if AI progress begins moving faster than human oversight.
The company said AI is already playing a larger role in AI development. Claude now writes most of the code used in Anthropic’s systems, and the company said the average engineer is producing about eight times more code per day than in 2024. Anthropic said Claude is also improving at research tasks, which could further accelerate AI development.
Why Anthropic Says a Pause Would Need Global Rules
Anthropic said any pause or slowdown in frontier AI development would need coordination among major AI companies and governments. A single company stopping on its own would not be enough, the company warned, because competitors could continue building more powerful systems.
A workable pause, Anthropic said, would require shared rules, verification and participation across countries. That makes the proposal difficult, especially as Washington and Silicon Valley remain focused on keeping the United States competitive against China in advanced AI.
President Donald Trump signed an executive order on Tuesday, June 2, 2026, allowing the federal government to access covered frontier AI models for up to 30 days before release to trusted partners.
Anthropic’s warning does not mean AI is already building itself without human control. But the company said the growing role of AI in coding and research shows why policymakers may need to act before recursive self-improvement becomes a reality.
