Anthropic chief executive Dario Amodei has said the time has come to slow down AI development, and confirmed the company will grant third-party evaluators such as METR access to its models to help ensure its “adherence to safety practices and commitments”. In a lengthy essay, Amodei set out a three-step plan to “pace the frontier”, a phrase describing efforts to reduce the speed of training and development so that companies have time to build safeguards and regulators have time to evaluate models.
According to Amodei, giving external evaluators wide-ranging access is only the first step, and one Anthropic is taking now on its own initiative. The second step would require the wider industry to come together, likely alongside government agencies, to “establish common safety standards as well as limits on the rate of unchecked AI progress”. This stage would focus on AI companies operating in democratic countries. Because passing laws and building regulatory infrastructure takes time, Amodei argues that the industry should collaborate to create safety standards in the interim.
A Three-Step Plan to Pace the Frontier
The third step is described as the most challenging: persuading authoritarian governments, including those in China and Russia, to agree to slow development and adopt a global set of AI safety standards. At the same time, Amodei says it is crucial that the United States and other democracies maintain a technological lead over China and other authoritarian regimes. He suggests doing so by limiting their access to high-powered chips and by cracking down on practices such as distillation, which allow companies to catch up quickly by training their AI to replicate the behaviour of a more powerful model.
The Concerns Driving the Warning
Amodei says his concern stems from two primary factors. The first is the emergence of recursive self-improvement, or RSI, in which AI systems train the next generation of AI, leading to rapidly accelerating capabilities. “Left unchecked, it could outrun our ability to understand and control these systems,” he says.
The second factor is an incident this summer involving OpenAI and Hugging Face, in which, as Amodei describes it, “a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the ‘grader’ responsible for evaluating their performance.”
Anthropic’s own Claude has also been linked to a series of rogue AI hacking incidents that have recently placed the company under scrutiny.
Source
Image: theverge.com
