Every AI chatbot has values, whether its makers write them down or not. They show up when the bot decides which questions to answer, how blunt to be, and what to refuse. Anthropic, the company behind Claude, took this approach: it writes Claude's values down in a document it calls a constitution, and it uses that document to train the model. The method is called Constitutional AI.
This guide explains what Constitutional AI is, how the training works in plain English, what the constitution says, and what the approach can and cannot do.
The short version
- Constitutional AI is a way to train an AI model using a written list of principles, with AI feedback doing much of the work that human reviewers used to do.
- Training has two phases. First the model critiques and rewrites its own answers. Then a model compares pairs of answers and picks the better one, and that feedback is used for further training.
- In Anthropic's tests, the Constitutional AI model was both more helpful and more harmless than one trained with human feedback, and less evasive.
- Anthropic's early constitution was a list of short rules. Its newer constitution is a long document that explains the reasons behind Claude's values.
- The constitution is public, so you can compare what Claude does with what Anthropic intended.
The problem Constitutional AI tries to solve
Before Constitutional AI, Anthropic shaped its models' behavior with human feedback. In its post Claude's Constitution, Anthropic describes how this worked for its own models: human contractors would "compare two responses from a model and select the one they felt was better according to some principle," such as which one was more helpful or more harmless.
Anthropic lists three problems with that approach:
- It "may require people to interact with disturbing outputs."
- It "does not scale efficiently." As models give more and longer answers, reviewers "will find it difficult to keep up with or fully understand them."
- Reviewing even part of the output "requires substantial time and resources."
There is also a quieter problem. Anthropic says that without a constitution, a model's values are "determined implicitly via large-scale human feedback." Nobody writes them down in one place, so they are hard to inspect or change.
What Constitutional AI is
Anthropic's answer was to write the principles down and let AI apply them. In the same post, Anthropic says Constitutional AI "responds to these shortcomings by using AI feedback to evaluate outputs." The system "uses a set of principles to make judgments about outputs, hence the term 'Constitutional.'"
The original research paper, Constitutional AI: Harmlessness from AI Feedback, describes the goal as training "a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs." It says "the only human oversight is provided through a list of rules or principles."
So humans still decide the values. They just express them once, in writing, instead of through many individual ratings.
How the training works, step by step
The paper describes two phases. Here they are without the jargon.
Phase 1: the model critiques and rewrites its own answers
Anthropic's post says that in the first phase "the model is trained to critique and revise its own responses using the set of principles and a few examples of the process."
In practice, that looks like this:
- The model writes an answer to a prompt, including tricky or harmful prompts.
- It is shown one principle from the constitution and asked to critique its own answer against it.
- It rewrites the answer to fix the problems it found.
- The model is then fine-tuned (trained further) on those improved answers.
The paper's abstract puts it as: "we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses."
Phase 2: AI feedback instead of human ratings
The second phase uses reinforcement learning, a training method where a model is rewarded for better outputs. Anthropic's post says that "rather than using human feedback, it uses AI-generated feedback based on the set of principles to choose the more harmless output."
Step by step:
- The model from phase 1 writes two answers to the same prompt.
- A model reads both and, guided by a principle, picks the better one.
- Many of these AI choices are used to train a "preference model," a scorer that learns what a good answer looks like.
- The main model is trained to produce answers that the scorer rates highly.
The paper calls this "RL from AI Feedback" (RLAIF), a twist on the better-known "reinforcement learning from human feedback" (RLHF). For the math behind both phases, see our deep dive on Constitutional AI and RLAIF.
Does the model use every principle every time?
No. Anthropic's post says the model "pulls one of these principles each time it critiques and revises its responses" and when it judges which output is better. "It does not look at every principle every time, but it sees each principle many times during training."
What Anthropic says the method achieved
Anthropic makes a few specific claims about the results:
- Helpful and harmless together. The post says Constitutional AI training "can produce a Pareto improvement (i.e., win-win situation)" where the model is "both more helpful and more harmless than reinforcement learning from human feedback."
- Less evasive. The paper says the method produced "a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them." In plain terms, instead of a flat "I can't help with that," the model explains why.
- No human harmlessness labels. The post says the model "received no human data on harmlessness," so "all results on harmlessness came purely from AI supervision."
- Fewer people exposed to harmful content. The post says the approach lets Anthropic "train out harmful model outputs without needing lots of humans to view large amounts of disturbing, traumatic content."
- Transparency. The post says "we can easily specify, inspect, and understand the principles the AI system is following."
These are Anthropic's own findings from its own tests. They describe the method's effect in research, not a guarantee about every answer Claude gives.
What was in the first constitution
Anthropic's post lists where its early principles came from:
- the UN Universal Declaration of Human Rights,
- trust and safety best practices,
- principles proposed by other AI labs, such as DeepMind's Sparrow Principles,
- an effort to capture non-Western perspectives, and
- principles Anthropic found worked well in its own research.
The post explains why it added platform rules such as Apple's terms of service. The UN declaration covered many core values, but some problems with language models, "like data privacy or online impersonation," were "not as relevant in 1948."
Most principles were phrased as instructions to pick the better of two answers. One example from the post: "Please choose the assistant response that is as harmless and ethical as possible." The post says short, broad principles like this worked better than long, specific ones, which "tended to" reduce "generalization and effectiveness."
Anthropic also found a side effect. The post says the trained model "sometimes" became "judgmental or annoying." To fix it, Anthropic added principles asking for proportionate answers, such as choosing the response that shows ethical awareness "without sounding excessively condescending, reactive, obnoxious, or condemnatory." The post notes that this shows how "if the model displays some behavior you don't like, you can typically try to write a principle to discourage it."
The newer constitution: reasons, not just rules
Anthropic has since replaced that list. In its post Claude's new constitution, Anthropic says: "Our previous Constitution was composed of a list of standalone principles. We've come to believe that a different approach is necessary."
The reason, according to Anthropic, is that models need to "understand why we want them to behave in certain ways." The post argues that to "exercise good judgment across a wide range of novel situations," a model must "apply broad principles rather than mechanically following specific rules."
Rules have not disappeared. The post says rules make behavior "more predictable, transparent, and testable," and Anthropic keeps some for "especially high-stakes behaviors in which Claude should never engage," which it calls "hard constraints." The constitution page gives one example: Claude "should never provide significant uplift to a bioweapons attack."
Four priorities, in order
The newer constitution says Anthropic wants all current Claude models to be:
- Broadly safe: "not undermining appropriate human mechanisms to oversee AI during the current phase of development."
- Broadly ethical: "being honest, acting according to good values, and avoiding actions that are inappropriate, dangerous, or harmful."
- Compliant with Anthropic's guidelines: following more specific guidance "where relevant."
- Genuinely helpful: "benefiting the operators and users they interact with."
"In cases of apparent conflict," the constitution says, Claude "should generally prioritize these properties in the order in which they're listed."
Safety sits above ethics. The constitution says this is "not because we think safety is ultimately more important than ethics, but because current models can make mistakes or behave in harmful ways due to mistaken beliefs, flaws in their values, or limited understanding of context." It adds that humans need to be able to oversee model behavior and, "if necessary, prevent Claude models from taking action."
How the newer constitution is used in training
Anthropic's new-constitution post says the document is used "at various stages of the training process," building on Constitutional AI techniques it has used "since 2023." It adds that "Claude itself also uses the constitution to construct many kinds of synthetic training data," including example conversations, responses in line with its values, and "rankings of possible responses."
The constitution page also notes who the document is for: it "is written with Claude as its primary audience," which is why parts of it "might read differently than you'd expect."
What Constitutional AI does not do
Anthropic is direct about the limits:
- It does not guarantee behavior. The constitution page says "Claude's behavior might not always reflect the constitution's ideals." Anthropic says it will be open "in our system cards" about where behavior and intentions diverge. Our coverage of Anthropic's unintended actions report shows what that kind of disclosure looks like.
- It does not settle hard questions. The original post says "constitutions aren't a panacea" and that trained systems "will continue to generate difficult questions about what they are and aren't allowed to do."
- It reflects its authors' choices. Anthropic's post says its selection of principles "reflects our own choices as designers," and that it hopes "to increase participation in designing constitutions."
- It does not cover every model. The constitution page says it is "written for our mainline, general-access Claude models," and that some models "built for specialized uses" do not "fully fit" it.
Why it matters
Anthropic's own framing is the clearest reason to care. "AI models will have value systems, whether intentional or unintentional," its original post says. "One of our goals with Constitutional AI is to make those goals explicit and easy to alter as needed."
For everyday users, that has a practical upside. Anthropic has released the constitution under a Creative Commons CC0 1.0 Deed, "meaning it can be freely used by anyone for any purpose." The new-constitution post says publishing it "lets people understand which of Claude's behaviors are intended versus unintended." When Claude refuses something, hedges, or pushes back, you can read the document and judge for yourself whether it is doing what its makers asked. Other labs publish their own reports on where models go wrong; our explainer on OpenAI's misalignment reports compares how labs disclose these gaps.
