OpenAI announces new AI safety program; researchers worry it won't be enough

AI safety researchers say OpenAI's new third party evaluator program falls far short of what the company promised, raising concerns about its effectiveness.The company announced Monday that staff from four AI safety and research organizations will become "embedded assessors" to "give people a clearer view of how labs are evaluating and managing [potential] risks.""Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable," CEO Sam Altman said at Semafor's Silicon Valley and the World conference on Monday.However, many AI safety researchers say the new program is a far cry from Altman's vow in September that independent evaluators would have "employee-like access" to internal company systems.Altman's promise came after details of the Hugging Face attack by OpenAI's autonomous systems shook the AI field.Worries of the technology's catastrophic risk captured public attention, and rival Anthropic said it would set up such a program."We are giving deep access to some of the most sensitive tools and models our researchers use...that are the most important for assessing key, relevant and timely safety claims," said Lama Ahmad, OpenAI's frontier assurances lead, on X.Prior to these new embedded evaluation programs, AI companies have brought in outside researchers to assess the capabilities of AI models and identify safety loopholes the companies have missed.The work of outside evaluators have already helped shed light on the companies' work, and have raised public awareness of safety risks that the technology poses.

One recent report about Anthropic using AI to significantly speed up model development has prompted concerns in the field about how close the top companies are to using AI to autonomously build even more powerful models, a stage known as recursive self-improvement.Some in the field worry that if this happens, the velocity of development could lead to catastrophic consequences for humani...

Read More 
PaprClips
Disclaimer: This story is auto-aggregated by a computer program and has not been created or edited by PaprClips.
Publisher: NPR News

Recent Articles