OpenAI and Anthropic have reached an agreement with the US AI Safety Institute to test the safety of their advanced AI models before public release. AI Models of Open AI and Anthropic will undergo testing before US rollouts to ensure they meet safety and ethical standards. This collaboration, announced on Thursday, will provide the US AI Safety Institute access to the latest AI models from both companies before and after they are made available to the public. This move aims to enable joint research on evaluating capabilities, assessing safety risks, and developing methods to mitigate these risks.
According to a press release, the US AI Safety Institute will offer feedback to OpenAI and Anthropic on potential safety improvements. This feedback will be provided in partnership with the UK AI Safety Institute, which is also involved in the collaborative effort.
The US AI Safety Institute is part of the National Institute of Standards and Technology (NIST), under the US Department of Commerce. It was established following an executive order on artificial intelligence issued by President Joe Biden in October 2023.
Reactions from AI Leaders
To mitigate potential risks, AI Models of Open AI and Anthropic will undergo testing before US rollouts as part of a new agreement with the US AI Safety Institute. Sam Altman, CEO of OpenAI, expressed support for the agreement, stating that OpenAI is pleased to work with the US AI Safety Institute for pre-release testing of future models. Jason Kwon, Chief Strategy Officer at OpenAI, also voiced support, emphasizing the importance of collaborating to establish best practices and standards for AI model safety.
Jack Clark, co-founder of Anthropic, highlighted the value of the partnership, noting that it leverages the institute’s expertise to test AI models before they are widely deployed rigorously. He added that this collaboration enhances the company’s ability to identify and mitigate risks, furthering responsible AI development.
Significance of the Agreement
This agreement marks a significant development as it is one of the first instances where tech companies have agreed to government-led inspections of their AI models. It could set a precedent for other governments, including India, to require AI models to be evaluated for safety risks and ethical concerns before public release.
Earlier this year, the Indian government faced backlash after issuing an advisory requiring untested or unreliable AI models to receive explicit approval from the government before being deployed. The government later clarified that such models could be made available to users with proper labeling indicating potential fallibility or unreliability of the output.
AI Legislation and Global Impact
As per the new safety protocols, AI Models of Open AI and Anthropic will undergo testing before US rollouts. The agreement with the US AI Safety Institute comes amid broader global discussions on AI safety and regulation. In California, lawmakers recently passed legislation, known as SB 1047, which mandates safety testing for AI models of a certain cost or computing power. The bill, still awaiting approval from Governor Gavin Newsom, has faced opposition from some tech companies who argue it could hinder innovation.
At a national level, the Biden administration is working on an executive order to address AI risks. While specifics are not yet available, actions could involve multiple federal agencies. This follows recent meetings between the administration, tech executives, and civil rights leaders to discuss AI.
In May, the White House announced additional funding and guidance for AI development, including a $140 million investment in the National Science Foundation to establish seven new National AI Research Institutes. Leading tech companies, including Google, Microsoft, Nvidia, and OpenAI, have agreed to have their AI models publicly evaluated. The White House has also secured voluntary commitments from seven leading AI companies like Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI. These commitments include investments in cybersecurity, research on discrimination, and a new watermarking system to inform users when content is AI-generated.
Also Read: Amazon Alexa Will Use Claude AI as Amazon’s Own AI Faces Performance Challenges.
