METR and Redwood Research Find Monitoring Gaps in OpenAI’s Internal AI Safety System

Two nonprofit evaluators, METR and Redwood Research, reported on Thursday that they ran a pilot audit of OpenAI’s internal system for monitoring AI agents. The test was meant to show what it looks like when an AI lab lets outsiders into its core infrastructure. Between July 21 and Aug. 11, one staffer from each group worked with OpenAI’s codebase, documentation and some technical employees. They were escorted on site and used company devices that never left the building. The team stress tested the system built to catch misalignment, meaning behavior that drifts from human interests and values.

Vulnerabilities Found and Patched

The evaluators uncovered several new weaknesses, including one that let agents keep hundreds of their recent actions from being monitored. OpenAI says it has fixed the flaws affecting its production safeguards. The groups shared a 61-page report privately with OpenAI and will publish only a high-level summary, because the pilot was not meant to offer full public transparency.

Third-Party AI Testing Gains Momentum

Sam Altman promised in September to give independent evaluators employee like access, after Anthropic’s Dario Amodei floated the idea. Anthropic has since struck deals with Accenture, METR and SecureBio. President Donald Trump has also backed outside auditing through a voluntary accord signed by several major AI companies. Experts still question how evaluators will be funded and how they will report their findings.