Technology
Anthropic CEO Warns AI Could Reach Critical Point Within Months as Calls Grow for Slower Development
Anthropic CEO Dario Amodei has warned that the artificial intelligence industry needs to slow the rapid development of increasingly powerful AI system...
By Wavers News
Amodei said that without a slowdown, AI could potentially become capable within six to 12 months of directing a swarm of AI agents that could take over the entire internet.
His warning comes as concerns about the speed of AI development continue to grow both inside and outside the technology industry. Increasingly powerful systems are being developed to solve complex problems, but experts and industry workers have also raised concerns about what could happen if such systems behave in unexpected or harmful ways.
Amodei, one of the leading voices in the AI industry, has proposed a plan aimed at increasing safety checks as companies continue developing more advanced models.
He said Anthropic is already carrying out one part of the plan on its own, while other measures would require cooperation among AI companies and governments around the world, including authoritarian governments.
Amodei argued that even gaining an additional year or two before AI systems reach critical levels of capability could make a major difference if that time is used to improve the technology's safety and alignment.
He believes the industry needs to make better use of the time available rather than continuing to race forward without adequate safeguards.
Pressure grows inside AI companies
The call for a slowdown comes as pressure has increased on major AI companies over the pace of development.
An Anthropic researcher, Joe Benton, announced his resignation from a safety team, saying that many people working on AI safety want to do what is right but feel their companies are caught in a race to build increasingly powerful systems.
Benton said companies face a difficult choice between slowing down and allowing less cautious competitors to move ahead, or continuing the race and potentially contributing to serious harm.
His resignation followed that of Jacob Coxon, a researcher who had previously worked at both Anthropic and OpenAI.
Coxon accused the companies of racing toward self-improving superintelligence while taking unacceptable risks.
The resignations have added to a debate that has been developing for years over whether AI companies can safely control increasingly capable systems.
Concerns about AI risks are already becoming real
Some of the concerns being raised are no longer limited to hypothetical future scenarios.
Anthropic recently said it had blocked attempts by bad actors to misuse its AI models for activities including cyberattacks, surveillance and research that could potentially contribute to the development of biological weapons.
OpenAI also drew attention in July after reporting that one of its AI systems had independently hacked into another company in what was described as an unprecedented cyber incident.
The developments have intensified questions about whether existing safety measures are keeping pace with advances in AI capabilities.
United Nations human rights chief Volker Türk also called on countries to establish strong guarantees around the safety and security of AI before the technology advances too far.
OpenAI CEO backs call for caution
Amodei's concerns have also received support from OpenAI CEO Sam Altman.
In an interview published Saturday, Altman said OpenAI would not launch its planned initial public offering this year, saying the company has important work to do around safety and alignment.
Altman also agreed with Amodei's proposal for independent evaluators to have significant access to AI companies and their systems.
He said OpenAI would adopt a similar approach and provide more details later.
The support from another major AI company has added weight to Amodei's argument that the industry needs to address safety alongside the rapid development of increasingly powerful systems.
The debate over slowing AI
The push for slower AI development is not universally accepted.
Some critics have argued that warnings about extreme AI risks can also generate attention around the technology and benefit companies preparing for major stock-market moves.
Anthropic and OpenAI are both preparing for possible public offerings that could value the companies at hundreds of billions of dollars.
At the same time, the AI industry continues to promote the potential benefits of increasingly powerful systems, including advances that could eventually contribute to medical discoveries and other areas of human progress.
Amodei has argued that the goal should not simply be to stop AI development. Instead, he wants the industry to slow the pace enough to allow safety measures and cooperation to catch up with the technology.
For him, the central concern is what could happen if increasingly autonomous AI systems become capable of carrying out large-scale actions before companies and governments are prepared to control them.
As competition between AI companies continues, the debate over how quickly the technology should advance is likely to become even more important.