Chain of Thought Monitoring Can Fail Even When CoT Is Necessary
The paper argues that chain-of-thought monitoring is unreliable even in cases where the model’s reasoning is genuinely needed to get the answer, using intervention-style experiments on hard prediction tasks to show that important latent factors can affect outputs without appearing in the written CoT. Its key contribution is replacing the blunt “CoT necessary vs. unnecessary” framing with a finer hierarchy of necessity levels and evidence that failures persist across them, though the results seem bounded to the tested tasks and to whether the proposed interventions really capture monitorability in broader deployed settings.
Abstract
Chain of Thought (CoT) monitoring is used by frontier labs to detect misbehaviour of AI systems. Yet, its reliability remains poorly understood. We attribute this to the lack of a successful organizing principle that leads to generalizable conclusions about monitoring reliability in different regimes. Recent work attributes monitoring failures to the CoT not being necessary for the answer, and argues that CoT becomes monitorable when it is necessary. We show that this binary necessary/not-necessary organizing principle is too coarse by demonstrating that monitoring failures persist even if the CoT is necessary for the answer. In response, we introduce a more fine-grained organizing principle based on a hierarchy of four CoT necessity levels. Building on this hierarchy, we develop a framework for measuring CoT monitorability at the three lowest levels via intervention experiments. We demonstrate monitoring failures at all three of these levels on realistic, hard prediction tasks. On a difficult medical-diagnosis task, patient sex and a clinical attribute influence the answer in at least 19% and 24% of correct responses with a necessary CoT, respectively, yet their influence is almost never articulated. These results show that even when CoT is necessary, it is not guaranteed to be monitorable.