Anthropic urges AI labs to pause, warns humans risk losing control

The article reports that Anthropic has suggested that top AI developers work out a joint method for halting progress on advanced systems, saying the pace of improvement raises the risk that people lose control. In a blog post published on a Thursday, the company said the world would benefit from being able to slow or briefly stop such work. Its in-house research institute will study the idea with partners and act to help build what a believable slowdown would require, though no specifics were given.

The post was written by cofounder Jack Clark and Marina Favaro, who leads the institute. It says AI is getting much faster at tasks such as coding and that, with enough computing power, a system might eventually design its own successor, known as recursive self-improvement. Anthropic sees benefits for science and healthcare but also a greater chance of losing control. A pause, the authors say, would give societal structures and alignment research time to keep pace. They want a mechanism that lets labs verify that rivals have really slowed down and that nobody secretly uses the arrangement to get ahead; without one, the least cautious players could catch up.

The article adds context. OpenAI published a report the day before arguing that democratic governments, not private firms alone, should set rules and that no single lab should decide the pace of innovation. A Trump administration order earlier in the week asked labs to voluntarily submit top models for government cybersecurity testing. Researchers at the University of Toronto, led by Nicolas Papernot, also showed how AI tools could produce an adaptive hacking worm. The article also recalls a 2023 call for a six-month halt, which had little success, and notes that Anthropic and OpenAI are racing toward share sales.