Anthropic, the most prominent AI company globally, has advocated for a worldwide slowdown in AI research and stressed that there is a high risk that humans will lose control of ever-evolving systems.
The company, known for its Claude AI language model, stated that it would agree to put an end to further advancements in AI technologies if other AI organizations also agree to do so. The move comes amid growing concerns in the industry over the rapid rate of AI advancement and potential implications.
According to an essay published by Anthropic, executives of the company including Marina Favaro and Jack Clark, noted that there is an urgent need for giving the world some more time to be able to cope up with advanced AI.
“If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing,” the executives wrote.
In its comparison to the arms control issue, Anthropic argued that there will only be a small window for introducing safety measures before AI becomes much more sophisticated.
This threat has significant credibility considering that Anthropic is not an outsider in the AI industry. On the contrary, it has established itself as one of the largest companies within the industry and has grown to be among the most valuable firms when it comes to artificial intelligence development. Some of its competitors include OpenAI and Google.
It is worth noting that Anthropic has taken the bold decision of keeping its most advanced AI system from being accessed by the public. The name of the AI system is Mythos and, according to Anthropic, it has the capability of conducting cyberattacks on a large scale if not put under any restrictions.
Recursive Self-Improvement and the Future of AI Development
One of the key concepts highlighted by Anthropic is the recursive self-improvement of AI. Recursive self-improvement refers to the stage at which AI systems would be able to develop new designs, improve coding, and conduct research autonomously.
According to Favaro and Clark, recent improvements suggest that AI development might be approaching this stage already. The authors state that staff members of Anthropic now write significantly more lines of code with the help of AI compared to just a couple of years ago. In addition, the researchers claim that AI-based systems are becoming increasingly efficient at producing ideas for studies and making plans for engineering works.
Developers of AI technologies regard this improvement as an indication of increased efficiency of work. Meanwhile, opponents of further advancements in this area are concerned about possible implications since the same qualities might make artificial intelligence systems hard to control.
It appears that this problem was mentioned by the representatives of Anthropic previously. The CEO of the company, Dario Amodei, believes that there is a high probability that advanced AI may lead to negative consequences if no preventive measures were taken.
Nevertheless, despite all these concerns, Anthropic admits that a global moratorium on development is nearly impossible to impose.
As the authors emphasize, there is strong competition between the United States and China, which considers artificial intelligence a strategically important invention. Any delay requires collaboration of multiple companies that develop AI.
Balancing Rapid Innovation, Safety, and Market Stability
According to Anthropic, participating organizations should have methods to ensure competitors stop working on advanced technologies. Contrary to nuclear weapons development, artificial intelligence research cannot be monitored as efficiently because of its capability to be done in data centers.
“If a slowdown simply allows the most reckless actors to catch up technologically, it might end up leaving everyone worse off,” Anthropic said.
The opinions voiced by Anthropic are not supported by all researchers, and some of them claim that the capabilities of AI technologies are always exaggerated. Also, there are concerns that the existential threat claims will result in regulations that will protect existing companies and make market entry more difficult for new competitors.
All of this comes amid concerns about the state of investor confidence in the sector.
After Broadcom provided sales estimates that disappointed investors, its shares plunged. The drop wiped out hundreds of billions of dollars from the market capitalization of many companies, including Broadcom, Micron, SK Hynix, and Arm.
Such reactions have reignited worries over whether excitement over AI might be driving the prices of tech shares up to unsustainable levels. Indeed, there is growing interest in finding signs that AI infrastructure demand can justify such extensive expenditures.
Currently, developments in AI are proceeding at breakneck speed. However, as shown by the recent statement issued by Anthropic, a growing gap has formed within the industry. As businesses rush to create ever more sophisticated artificial intelligence programs, it becomes impossible not to consider questions of safety and regulation.
There are still questions about whether an overall slowdown is imminent. It is clear, however, that some of the major players in the field of AI development are already warning about the dangers of unregulated growth.
