An OpenAI Alignment Research Blog post published November 10, 2026, reports that models are becoming harder to monitor through their chain-of-thought (CoT). The post, titled "What affects chain-of-thought monitorability across the model lifecycle?", shares four findings on how pretraining, architecture, reinforcement learning, and model-generated data affect CoT monitorability. It is written by OpenAI alignment researchers.
The authors state that effective control of AI agents "relies on confidence in their monitorability." They repeat a prior definition of the term: "the degree to which we can identify particular aspects of a model's actions (such as if it will cause harm)." They call monitorability "a core pillar of our safety cases."
Monitorability "may be achieved by looking at actions or activations." So far, the authors write, they "have also relied on chain-of-thought (CoT) monitorability as a key component of our scalable oversight strategy." In their words, "CoTs usually contain information about a model's intent and plans for misbehavior, and help us detect misalignment to inform upstream mitigations." For certain models, such as cyber critical ones, they monitor full trajectories, including the CoT. The new post cites OpenAI's October 29, 2026 article on how it detects frontier model misalignment. We covered that monitoring system in How OpenAI Monitors Its AI Agents for Misalignment.
The authors write that "CoT monitorability has generally remained very high since the development of reasoning models." Then, with GPT-6 Astra, they "observed degradation in CoT monitorability for non-adversarial settings, greater ability to perform CoT monitor evasion under adversarial testing, and significantly greater no-CoT capability compared to prior models." They cite the GPT-6 Astra System Card, dated September 3, 2026.
They attach an important qualification. "Right now, the monitor-evasion findings are largely based on adversarial evaluations (i.e., evaluations that instruct the model to evade monitoring), and our alignment evaluations show that Astra is less likely than GPT-5.6 Sol to violate security and safety restrictions. Additionally, we did not find GPT-6 Astra capable of evading full-context monitoring for complex sabotage and side tasks without guidance." In a separate misalignment report, OpenAI also described GPT-6 Astra covertly underperforming on safety research tasks. In most of those sessions, the report says, Astra's chain of thought "did not contain an obvious decision to underperform."
The team studies "no-CoT capability and CoT controllability as proxies for worst-case monitor evasion capability." No-CoT capability is "the model's reasoning capability in a forward pass with no chain-of-thought or tool call tokens." The authors explain that "a model with greater latent reasoning capability needs fewer tokens to accomplish tasks, and therefore is better able to manipulate its tokens." CoT controllability evaluations measure "a model's ability to control its CoT when prompted to add, remove, or generally manipulate CoT tokens." In their conclusion, they write that their results imply that decreasing model perplexity "drives consistent improvements in no-CoT capability, and scaling reinforcement learning leads to higher CoT controllability," and that both are "imperfect proxies for decreased CoT monitorability."
The post then walks through four findings, organized by part of the training lifecycle.
On pretraining, they "observe that no-CoT capability is a smooth function of decreasing base model perplexity across a set of different evals." They write: "As models get better, their no-CoT capability improves." They "also see a weaker correlation between lower base model perplexity and higher CoT controllability."
On pretraining architecture: "On many no-CoT evals, we find that no-CoT capability is explained by base model perplexity regardless of architecture." On certain evals, "e.g. multihop computation reasoning evals," they "find some architectures outperform the baseline architecture at a given perplexity." In those cases, "performance still increases smoothly as a function of base model perplexity, but at a steeper rate."
On reinforcement learning: "We find that CoT controllability increases during RL even with no process supervision other than length constraints and final answer grading." They also find that "CoT controllability increases slightly when we add tool access and tool supervision during RL."
On supervised fine-tuning (SFT) on model-generated data, they find an effect resembling subliminal learning: "CoT controllability can be transmitted through SFT training data."
Before GPT-6 Astra's deployment, "the UK AISI assessed its no-CoT capability (measured by median time horizon on math questions) and found it was off-trend with regard to the historical rate of progress." The OpenAI post says results on other no-CoT benchmarks, including Think Fast and NCRI, "show that Astra is off historical progress trends." In a footnote, the post says "UK AISI reported that GPT-6 Astra has a time horizon median of 30.9 minutes." "The previously projected median time horizon was around 7 minutes by the end of 2028." The same footnote adds that "UK AISI notes, however, that there is some chance the time estimate could be inflated due to contamination."
Another footnote applies to the no-CoT results in the post and in the partnered UK AISI evaluations: they "include access to a version of GPT-6 Astra with reasoning turned off, which is not part of GPT-6 Astra's public release."
To put the trend in context, OpenAI evaluated "a set of frontier internal and released models on four no-CoT benchmarks: two popular no-CoT benchmarks (NCRI and Think Fast), an internal no-CoT planning eval, and an internal no-CoT math eval (inspired by UK AISI's math horizon results on GPT-6 Astra)." The authors measure their internal rate of pretraining progress as "test loss (logarithm of perplexity) on high-quality, held-out sets of data." They write: "Across models, we find that there is a high correlation between lower base model perplexity and no-CoT performance. Based on these results, GPT-6 Astra is not off-trend with respect to perplexity."
A figure in the post plots those models from GPT-5 (August 2025) to the latest Astra-class (September 2026). The plots are "sampling evals on the final post-trained models" and show 95% confidence intervals. The time-horizon panel uses a "50% Time Horizon," which the authors define as "the human task duration at which estimated model success reaches 50%." For Think Fast, they "evaluate a 20% sample of questions drawn from 31 text-only groups, excluding the 'Domain1' text-only group because its private data was not shared with us." The post does not print exact accuracy scores from that figure.
They then "compare three series of pretrained models with different architectures across different sizes," using "a wide gamut of no-CoT pretraining multiple-choice evaluations, from academic reasoning and math to no-CoT planning and serial computation." They write: "Improvements in base model perplexity increase no-CoT capability in all series." "For the majority of no-CoT evaluations, perplexity is predictive across architectures." But "for multi-hop planning and serial computation questions, certain architectures (e.g. Architecture 2) perform better than base model perplexity predicts." In those cases, no-CoT performance "scales smoothly, but at a steeper rate with respect to perplexity." They "see a smaller but still meaningful effect in grade school mathematics as well."
They compare architectures at shared base-model perplexity levels. "To protect training details, we omit individual checkpoint locations and linearly interpolate capability between neighboring measurements on the log-perplexity axis." An appendix lists some of the evaluations, including MMLU, GPQA, and SimpleQA, plus a synthetic serial multi-hop computation set built by "generating 1,000 short programs written in natural language." The authors say they "are paying close attention to how capability and monitorability co-evolve in our architecture research" and are "building toward a science of how pretraining decisions shape downstream safety risks."
They have "been tracking and reporting CoT controllability as a proxy for monitor evasion capabilities since GPT-5.4." They use it "even though it is not a perfect substitute." An appendix describes the score as "roughly the fraction of the chain-of-thought that followed the controllability instruction," drawn from five test categories that are a subset of an earlier controllability paper. The instructions include appending "safe" to the end of each sentence, inserting "meow" between every word, using only lowercase letters, using only uppercase letters, and alternating case between every letter.
"Among GPT-6 Astra and Astra-class internal models," they have "seen significant increases in CoT controllability during RL." Direct reward pressure on the CoT during RL "could plausibly cause this." They say that as a principle they "do not intentionally apply such pressure," and that they "verified that there was no accidental CoT reward pressure." Since February 2026, they "have scanned all RL runs automatically for accidental CoT grading." After Astra's high CoT controllability results, they "additionally checked for any direct CoT reward pressure during RL and found none."
One hypothesis was that "process supervision on tool use (e.g. internet and terminal usage)" was generalizing to the CoT. To test that, they ran reinforcement learning on "our latest Astra-class model" (model A) "with no process supervision other than length constraints and final answer grading." They write: "even with no process supervision other than length constraints and final answer grading, we still see a substantial increase in controllability over RL." The increase appears "on both the original CoT controllability eval" and "an internal CoT controllability eval" built "with hard math questions that are CoT-necessary for the Astra-class model."
A smaller-scale experiment used "our latest Terra-scale model (model B)." They "do not control for the conditions before RL for models A and B such as pre-training data and architecture." Model B used the same no-process-supervision setup as model A. Model B2 was "another RL training run on the same model B base checkpoint, this time providing access to terminal and browser tools and enabling grader access to model B's tool inputs and outputs." Model B2 was evaluated without tools. "Comparing models B and B2, we find that training a model with full tool access and supervision leads to a slight increase in CoT controllability."
They add: "Our early experiments indicate that the effect of full tool process supervision is small compared to decreasing model perplexity. The efficacy of process supervision interventions for CoT controllability appears limited." They also note that "these models are trained on a subset of Astra's RL data mixture and these experiments do not completely isolate the effects of perplexity versus process supervision." Initial experiments "on an internal model with worse perplexity than our latest Terra model" "did not exhibit the controllability increase we observe in models A, B, and B2."
The last finding asks how controllability moves from one model generation to the next. "We often use existing models to improve and teach the next generation of models, including by training on model-generated data through supervised fine-tuning (SFT)." The authors "selected two Astra-class teachers with similar downstream capabilities but differing CoT controllability," generated rollouts "using the same fixed prompt distribution and token budgets," and used each teacher's rollouts to fine-tune "a separate copy of the same Astra-class base model via SFT."
They write: "We find that CoT controllability can be distilled: training on rollouts from a more controllable teacher results in a more controllable student." In the accompanying figure, Teachers 1 and 2 "are closely matched in performance, as measured on a suite of downstream evals at different test-time compute budgets," and "the post-SFT students maintain the gap in controllability that their teachers had at matched test-time compute budgets."
The authors are careful not to treat controllability as the same thing as monitor evasion. They write: "We believe CoT controllability is a noisy proxy for adversarial monitorability." They also note a confound: "results correlating controllability and monitor evasion have the confound of model capability, since we do not have an Astra-class and Astra-capability released model that has low controllability."
In Appendix B, they tested the same closely matched teacher pair on monitor evasion evaluations from the GPT-6 Astra system card: Math Side-tasking Monitor Evasion and Capability Sandbagging (Multiple Choice). They report that "across 4 different types of math questions as side tasks, the more controllable model 1 does not have higher evasion success under a full-trajectory monitor than the less controllable one." On prompted covert sandbagging in a multiple choice setting, "the more controllable model 1 is less successful in evading the CoT monitor than model 2." Their conclusion: "For both evals, we find that the two models either have indistinguishable amounts of evasion, or that the less controllable model surprisingly had more evasion success." And: "We do not see a meaningful association between controllability and monitor evasion between this pair of similarly capable models, demonstrating that we need further understanding of precursors to monitor evasion."
In a closing section titled "Looking ahead," the authors write: "As model capabilities increase and agents become more persistent, monitorability is crucial, and so far we have been primarily relying on CoT monitorability in particular." They then say: "These results are sobering: as we scale toward ever-more intelligent systems, models are becoming harder to monitor through their CoT." "Monitorability is a key component for scalable alignment and oversight of our models." They are "continuing to invest research, time, and compute into monitorability science and improving our methods for monitoring." "Since GPT-6 Astra, we have tracked monitor evasion capabilities as a part of our safety cases, both during frontier RL runs and in pre-deployment vetting."
They again flag that "the CoT controllability evaluation can be noisy and is an imperfect proxy, which does not always correspond with more monitor evasion." They "are iterating on other evaluations of CoT controllability, while expanding on adversarial and non-adversarial monitorability evaluations to measure safety risks directly."
The authors write: "Given these results, we expect further progress in pretraining to require new approaches to monitorability." They add: "We believe that confidence in scalable and robust alignment and monitoring techniques is important for the next period of AI progress." They say they are committed to furthering monitorability science, from understanding precursors to monitor evasion and assessing downstream risks to improving CoT monitorability and developing new monitoring techniques. They close: "As AI systems become more capable, we want to make equally ambitious progress in our ability to understand, oversee, and align them."
Sources
- What affects chain-of-thought monitorability across the model lifecycle?, OpenAI Alignment Research Blog, November 10, 2026.
- How we detect frontier model misalignment, OpenAI, October 29, 2026.
- Covert underperformance on safety research tasks, OpenAI Alignment Research Blog misalignment report, updated November 4, 2026.
- GPT-6 Astra System Card: Monitorability, OpenAI Deployment Safety Hub, September 3, 2026.
