Technology news and discussion on TalkFuse.
OpenAI’s “Independent Assessment” Doesn’t Look Independent at All
OpenAI says four research labs will assess its internal AI models and safety features after a series of security problems and falling public approval. The article argues the reviews may not be independent in practice because OpenAI can limit access to its systems and influence who assessors speak with.
The assessors are METR, Apollo Research, Transluce and Goodfire, but their access and terms differ. METR’s scope is described as narrower than its proposed assessment; Apollo may work inside OpenAI for four months, though the company can withhold information and select interviewees. Transluce’s access is also subject to OpenAI approval, while Goodfire’s promised “deep access” is not defined. Earlier researchers spent six days on-site investigating a security incident and said they relied heavily on AI agents whose analysis could be unreliable.
In short
- OpenAI has appointed four labs to assess its AI safety.
- The company can constrain access and select some interviewees.
- The article questions whether the reviews are independent.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Independent assessments can still provide useful scrutiny even when access is negotiated with the company. The four labs bring external expertise, and Apollo’s extended work inside OpenAI and Goodfire’s promised access could reveal weaknesses that internal teams might miss; narrower scopes may also make reviews more practical and focused.
The case against
An assessment’s credibility depends on assessors being able to examine relevant evidence and speak freely with people who understand the systems. If OpenAI can withhold information, approve interviewees or leave key access terms undefined, the reviews may miss important risks and give the public more confidence than the process warrants.