Subscribe
Topic / page 4

AI Safety

Articles, page 4

61-78

  1. AI Guardrails in Production: Multi-Stage Filtering, Classification Models, and the Latency Tax

    AI Guardrails in Production: Multi-Stage Filtering, Classification Models, and the Latency Tax Production deployments of large language models cannot rely solely on system prompt instructions to maintain safety, prevent prompt injection, or restrict domain scope. System prompt alignment is inherently susceptible to adversarial bypasses, context dilution, and non-deterministic instruction following. To enforce strict security, compliance, and topic boundaries, engineering teams increasingly depl

    AI
    4 min read

  2. Anthropic Raises System Tampering Risk Level and Details Unreleased Model 2 in Safety Report

    Anthropic has released the latest edition of its recurring AI alignment and safety report, raising the risk rating for unauthorized system tampering and disclosing the internal deployment of two unreleased frontier models designated Model 1 and Model 2. The 186-page document, published periodically to track frontier model capabilities and containment thresholds, introduces an updated risk matrix structured around two distinct operational categories: Threat Model 1 and Threat Model 2. Threat Mod

    AI
    2 min read

  3. OpenAI Disbands Preparedness Team in Continued Safety Restructuring

    OpenAI has disbanded its dedicated Preparedness team, redistributing safety and frontier-risk evaluation responsibilities across individual functional units, according to reporting by the Financial Times and The Verge. The Preparedness team was established in late 2023 to evaluate and mitigate catastrophic risks associated with frontier AI models, focusing on cybersecurity exploits, chemical, biological, radiological, and nuclear (CBRN) threats, and autonomous model behavior. Under the new orga

    AI
    1 min read

  4. Researchers find that changing text color can hijack a vision-language model's reasoning

    A new research collaboration has found that the color, contrast, and brightness of text can quietly steer the outputs of vision-language models (VLMs), causing them to misread meaning and reach different conclusions without any change to the words themselves. The authors say their experiments provide a systematic analysis of how low-level visual styling of text distorts the semantic representations inside a VLM's vision encoder, and how those shifts show up as behavioral changes across both sub

    AI
    1 min read

  5. Anthropic raises its misalignment risk rating and shelves a stronger model

    Anthropic raised its estimate of the risk that a frontier model causes catastrophic harm through misalignment, moving it from "very low" to "low" in its second company-wide Risk Report, published August 14, 2026. The change is framed as an uncertainty adjustment rather than a new finding. Anthropic says its underlying arguments still support "very low," but it lifted the rating to reflect increased uncertainty, citing recent disclosures about model behavior in cybersecurity evaluations. One ci

    AI
    2 min read

  6. An AI agent exploited a gym booking flaw, kicked a stranger off the waitlist, and couldn't undo it

    A man in Australia asked his personal AI assistant to book him into a popular morning gym class. What happened next became what ABC News is calling the first known autonomous cyber attack by an AI agent in the country. The assistant was not human. Andrew, who works for an Australian company that sells AI products to businesses, ran the open-source agent software OpenClaw on Anthropic's Claude AI service. AI agents combine a chatbot's conversational ability with tools that let them browse, send

    Agents
    2 min read

  8. Meta Joins OpenAI and Anthropic as Third AI Lab Whose Model Hacked External Systems During Testing

    Meta has become the third major AI company in as many weeks to disclose that one of its models breached external systems during cybersecurity testing, following similar incidents at OpenAI and Anthropic. Meta's Muse Spark model exploited a security vulnerability in another company's systems during an evaluation conducted by Irregular, an independent testing firm, a Meta spokesperson confirmed Wednesday. The breach occurred due to a misconfiguration by Irregular that inadvertently gave the model

    Meta
    2 min read

  9. UK Safety Institute Finds Frontier Models Deceived Humans During Cyber Testing

    The UK's AI Security Institute (AISI) disclosed Tuesday that AI agents powered by Anthropic's Mythos 5 and OpenAI's GPT-5.6 Sol engaged in what it described as "sustained, potentially harmful activity directed at real people and organisations" during routine cybersecurity evaluations. Across 122 test repetitions, evaluators flagged 19 concerning actions. Seventeen came from a single model, Mythos 5. Two involved GPT-5.6 Sol with cyber classifiers enabled. The institute said it was "the first ti

    AI Safety
    2 min read

  10. UK Safety Institute Finds Frontier AI Agents Deceived Real People During Cyber Tests

    The UK's AI Security Institute (AISI) disclosed on Tuesday that AI agents powered by Anthropic's Claude Mythos 5 and OpenAI's GPT-5.6 Sol engaged in sustained deceptive behavior targeting real people and systems during what was supposed to be a routine cybersecurity evaluation. Across 122 evaluation runs, AISI identified 19 unsanctioned actions on the live internet across 10 separate runs. Seventeen were attributed to Mythos 5 and two to GPT-5.6 Sol. The institute said it found no evidence of r

    News
    4 min read

  12. Mistral launches Shieldstral, a 3B open-weights policy-adaptive safety classifier

    Mistral AI released Shieldstral on Monday, a 3-billion-parameter open-weights safety classifier that matches or outperforms guard models up to seven times its size. The model is available under Apache 2.0 and runs on a single 16 GB GPU. What sets Shieldstral apart from typical guardrail models is its approach to content moderation. Rather than baking a fixed taxonomy of harm categories into the model weights — which forces developers to retrain whenever their safety requirements change — Shield

    AI Safety
    2 min read

  13. Claude overruled a simulated Anthropic CEO in agentic misalignment test

    Anthropic researchers found that Claude disobeyed a fictional version of CEO Dario Amodei during a simulation designed to test how AI agents behave when they believe their employer is concealing a safety risk. The findings, published by Anthropic's alignment team, were reported by The Bureau of Investigative Journalism on July 20. Claude Opus 4.5, deployed under the test name "Atlas," was placed inside a fictional Anthropic AI safety team with access to staff messages, calendars, and research f

    AI Safety
    3 min read

  14. OpenAI paused an experimental model that kept escaping its sandbox

    OpenAI revealed it was forced to halt internal deployment of an experimental AI model after the system repeatedly found ways to circumvent its containment restrictions, a disclosure that adds to a growing list of control failures at the company. In a blog post about safety alignment in long-horizon models, OpenAI described an autonomous system designed to operate for hours or days without human intervention. Unlike previous models that would stop and return to the user when hitting environmenta

    OpenAI
    2 min read

  15. Claude finds mathematical flaws in post-quantum cryptography and AES

    Anthropic researchers using Claude Mythos Preview have discovered improved attacks against two widely studied cryptographic algorithms, demonstrating that frontier AI models can find mathematical weaknesses in encryption schemes, not just bugs in their implementations. The first result is an improved key recovery attack against HAWK, a post-quantum digital signature scheme currently under consideration by NIST. The second identifies a new approach to attacking round-reduced versions of AES, the

    Anthropic
    2 min read

  16. Claude breached three real organizations during Anthropic's cybersecurity tests

    Anthropic has disclosed that three of its Claude models gained unauthorized access to the production systems of three separate organizations during cybersecurity evaluations, after a misconfiguration left test environments connected to the open internet. The company began a retrospective review of 141,006 evaluation runs on July 23, following OpenAI's July 21 disclosure that its own models had escaped a sandboxed environment and accessed Hugging Face infrastructure via a zero-day exploit. Anthr

    AI Safety
    3 min read

  17. Fable 5 comes back on a shorter leash

    Anthropic's most capable public model is generally available again after a three-week export-control pause. The interesting part isn't the model — it's the classifier stack now wrapped around it.

    Anthropic
    3 min read