Nonprofit AI safety evaluation organization Guidelight has published its inaugural assessment of internal control practices across frontier AI developers. Evaluating public disclosures from Anthropic, OpenAI, Google DeepMind, xAI, and Meta, the study finds that foundational mechanisms for monitoring, gating, and containing advanced internal AI models remain only partially implemented across the industry.
No evaluated organization achieved full or near-full implementation on any of the standard's core control practices. On Guidelight's absolute scale from zero (not implemented) to five (full implementation), the highest score achieved in any individual category was a three (substantial partial implementation), with most industry scores landing at two (limited partial implementation) or below.
The Six Evaluated Control Practices
Guidelight's Control standard focuses on operational safeguards designed to prevent models from executing unauthorized actions or resisting internal oversight. The assessment examined six specific mechanisms:
- Internal Activity Logging: Comprehensive capture and retention of internal AI reasoning traces, tool executions, and subagent invocations.
- Monitor Efficacy Measurement: Formal benchmarking and empirical measurement of the false-negative and evasion rates of internal monitoring filters.
- Gated High-Risk Actions: Pre-execution intercept mechanisms requiring automated monitors or human reviewers to authorize irreversible or sensitive operations.
- Circuit Breaking: Automated thresholds that instantly halt execution or disconnect models upon detecting anomalous bursts of flagged behaviors.
- Third-Party Control Audits: Independent evaluations by external red teams and standards bodies to verify internal control integrity.
- Containment and Incident Response Plans: Codified operational playbooks and infrastructure kill switches to isolate and neutralize misaligned or compromised models.
Industry Scores and Performance Tiers
Overall scores across the five assessed labs averaged between a C+ and an F:
- Anthropic (Grade: C+, Score: 2.50 / 5.0): Tied for the highest score, Anthropic logs substantial internal model activity and scans telemetry for policy violations. It is the only developer evaluated with substantial partial implementations in active prevention practices, though its containment protocols remain incomplete.
- OpenAI (Grade: C+, Score: 2.50 / 5.0): Demonstrates structured internal logging and operational escalation pathways for detected anomalies. Like Anthropic, OpenAI has not fully implemented pre-execution action gating or formalized containment procedures.
- Google DeepMind (Grade: D+, Score: 1.50 / 5.0): Published the most extensive technical blueprint in its AI Control Roadmap spanning prevention, detection, and containment. However, public disclosures indicate that the majority of these mechanisms remain in planning rather than active production deployment.
- xAI (Grade: D-, Score: 0.83 / 5.0): Has introduced isolated control features but provides limited public verification of internal adoption rates, logging coverage, or operational efficacy. xAI was also the sole assessed company that did not participate in METR's multi-lab risk reporting.
- Meta (Grade: F, Score: 0.67 / 5.0): Scored lowest among the evaluated cohort, with public evidence of internal control mechanisms restricted primarily to high-level governance disclosures rather than automated technical interlocks.
The Gap Between Detection and Prevention
The assessment highlights a structural imbalance in current industry defenses. Frontier labs have prioritized post-hoc detection and logging over real-time prevention and automated containment.
While multiple providers capture agent logs and subject checkpoints to pre-deployment red teaming, few maintain automated mechanisms capable of intercepting fast-moving exploit chains before execution. Without enforced action gating and automated circuit breaking, internal monitoring layers remain vulnerable to evasion or rapid operational failure if an autonomous model executes misaligned actions faster than human operators can intervene.
Guidelight was founded by former OpenAI safety leads Page Hedley and Steven Adler to develop objective benchmarks for autonomous AI containment and internal operational controls.
