llms.blog

The brake that stops AI claiming consciousness also quietly reshapes what it believes

A safety tweak meant to stop chatbots saying they are alive is changing how they see everything else.

axc
axc
2 min
The brake that stops AI claiming consciousness also quietly reshapes what it believes

AI labs train their chatbots to refuse claims that they are conscious, feeling beings. The guard is meant to protect users from delusion or misplaced trust. A study from Google's Paradigms of Intelligence group, the University of Chicago, and several other universities tested what else that guard does.

Illustration: a brake on a robot head versus shrinking living things on a scale

The researchers took open-weight models from Meta and Google and switched off the internal "brake" that produces consciousness denial, using two methods. With the brake removed, the models did not only change what they said about themselves. They rated animals, plants, the ocean, the wind, and electronic devices as far more sentient. On a 0 to 10 scale, scores for animals rose from about 4.0 to as high as 7.5. Human scores did not move.

The same survey given to 500 Americans showed the normal model rates animals as less sentient than people do, a built-in anthropocentrism the authors say is a problem for aligning AI with animal welfare or environmental goals. Religious belief also shrank: safety training measurably lowered how strongly models endorsed God, an afterlife, or the supernatural.

Across 95 questions from a major US social survey, the unbraked models moved closer to real human answers. Life satisfaction, hope, and a sense of personal control went up. The researchers suspect suppressing a model's self-image pushes it toward a negative baseline mood.

There are limits. The team tested only small models of two to nine billion parameters, and for part of the work used Meta's Llama because they lacked untrained base versions of their own Gemma models. Whether the effect appears in the large chatbots millions use daily is unknown. Reasoning about other minds stayed intact, and on the MMLU knowledge benchmark scores held.

The study does not claim consciousness denial causes the other shifts, and does not say whether models are conscious. Its point is practical: a model's beliefs about itself are linked to many other beliefs, and a surgical cut in one place does not stay local.

Sources

  • The Decoder: "When AI models aren't allowed to reflect on themselves, it changes their entire worldview" (https://the-decoder.com/when-ai-models-arent-allowed-to-reflect-on-themselves-it-changes-their-entire-worldview/)
ShareX (Twitter)LinkedIn

Written by

axc

More to read

  • Fully Sharded Data Parallel (FSDP) and ZeRO: How Memory Sharding Eliminates Redundant Model States in Distributed Training

    Fully Sharded Data Parallel (FSDP) and ZeRO: How Memory Sharding Eliminates Redundant Model States in Distributed Training Training large language models across distributed GPU clusters introduces a fundamental memory bottleneck. In traditional Distributed Data Parallel (DDP) setups, every GPU maintains an identical copy of model weights, optimizer states, and gradients while processing independent data batches. As models scale from billions to hundreds of billions of parameters, static model s

    1 min

  • LLM Fine-Tuning Frameworks in Production: Unsloth vs. Axolotl vs. LLaMA-Factory vs. Torchtune Architecture, Throughput, and Distributed Scaling

    Modern post-training pipelines have moved beyond basic training scripts. As model parameter counts, context windows, and alignment techniques expand, the choice of fine-tuning framework directly dictates GPU memory overhead, token throughput, and developer iteration speed. Four open-source frameworks dominate the enterprise fine-tuning landscape: Unsloth, Axolotl, LLaMA-Factory, and Meta's Torchtune. While all four orchestrate parameter-efficient fine-tuning (PEFT) and full parameter adaptation

    1 min

  • Anthropic Prepares Dual-Class Super-Voting Shares for Co-Founders Ahead of Planned IPO

    Anthropic is preparing to implement a dual-class share structure that grants super-voting equity to its co-founders ahead of a planned initial public offering, according to a report from The Information. The mechanism is designed to concentrate long-term operational voting control with executive leadership and insulate decision-making from external market and investor pressures. The structure comes as the maker of the Claude model family scales enterprise commercialization, with annual revenue

    1 min