Anthropic, the company behind the Claude chatbot, has called for greater coordination between the world’s most advanced AI developers and suggested there should be a mechanism that allows the industry to temporarily slow down or pause development if necessary.

In a blog post published this week, the company argued that rapid progress in AI capabilities could eventually lead to systems that are able to improve themselves without direct human involvement – a milestone known as “recursive self-improvement”.

According to Anthropic, such technology could unlock major breakthroughs across science, healthcare and other industries. However, it could also increase the possibility of humans losing meaningful control over increasingly powerful AI systems.

“The world should have the option to slow or temporarily pause” development if risks begin to outpace society’s ability to manage them, the company said.

A growing debate within the AI industry

Anthropic’s proposal highlights a growing divide among AI companies over how the technology should be governed.

While Anthropic is advocating for a coordinated mechanism that could enable developers to pause progress if required, rival OpenAI has argued that decisions about AI development should ultimately be made by governments rather than individual companies.

In a report released this week, OpenAI said that “democratic governments – not private companies acting alone – must ultimately determine the rules, safeguards and accountability mechanisms” surrounding advanced AI.

The debate comes as AI models continue to become more capable at carrying out complex tasks, including software development and coding, with minimal human input.

Anthropic believes current trends could eventually produce systems capable of designing and building more advanced versions of themselves, accelerating technological progress at a pace that may become difficult to supervise.

Fresh security concerns

The warning also follows separate research from the University of Toronto, where researchers recently demonstrated how AI could be used to create a new type of self-adapting cyber worm.

Unlike traditional malware, the AI-powered system was able to modify its hacking strategies as it spread across connected devices, potentially allowing it to compromise large networks more efficiently.

Lead researcher Nicolas Papernot said the findings show that AI-related security risks are not limited to the largest and most sophisticated models.

“Anything connected to the internet is now at risk because of how low the cost has become to mount these cyberattacks,” Lead Researcher Nicolas Papernot warned.

Researchers say increased collaboration between governments, technology companies and academic institutions will be essential as AI-powered cyber threats continue to evolve.

Can a global pause actually work?

One of the biggest challenges facing any proposed slowdown is enforcement.

Anthropic argues that any future pause mechanism would need to allow competing AI labs to verify that rivals had genuinely slowed or halted development, preventing less cautious players from secretly gaining an advantage.

Without international cooperation, the company warns that responsible developers may feel pressured to continue accelerating simply to avoid falling behind competitors.

The discussion is not entirely new. In 2023, several technology figures, including Elon Musk, backed a proposal calling for a six-month pause on advanced AI development to allow time for safety measures to catch up. The initiative ultimately gained little traction.

The race continues

Despite growing concerns about safety, AI development shows few signs of slowing down.

Anthropic and OpenAI remain locked in an increasingly intense race to build and commercialise more powerful AI systems, while governments around the world continue to grapple with how best to regulate the technology.

For now, Anthropic’s proposal is less about an immediate shutdown and more about ensuring the option exists before AI capabilities reach a point where society has little room to react.

The question facing the industry is whether those safeguards can be established before the technology advances beyond them.