Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
The paper argues that LLM safety needs “vigilance” beyond rote refusal rules, and shows that fine-tuning on “cunning questions” with misleading premises or subtle inconsistencies can make models better at spotting latent risk and resisting out-of-distribution jailbreaks. The interesting bit is the transfer claim: non-safety reasoning traps appear to improve safety behavior and complement standard alignment pipelines, though the gains reported are fairly modest and the evidence is limited to the evaluated backbones/benchmarks plus conditional analysis rather than a general guarantee.
Abstract
Safety alignment teaches large language models (LLMs) to recognize harmful requests and reject risky instructions. Yet aligned models can fail when harmful intent is concealed within seemingly benign contexts. Robust safety therefore requires both knowledge of safety boundaries and \textbf{vigilance}: the ability to detect unusual premises, misleading reasoning, and latent risks beneath surface-level semantics. Vigilance requires models to scrutinize a request's underlying intent and assumptions before acting. To cultivate this capability, we introduce \textbf{cunning questions}, which are not necessarily safety-related but contain misleading premises, atypical reasoning, or subtle inconsistencies. We hypothesize that learning to look beyond such reasoning traps can transfer to safety-critical scenarios. Experiments show that Cunning training improves robustness to out-of-distribution jailbreak attacks and strengthens subsequent safety fine-tuning. Furthermore, augmenting an existing state-of-the-art safety alignment pipeline with Cunning establishes a new state of the art across our evaluated settings, reducing mean ASR across nine backbone--benchmark combinations from 17.40\% to 15.05\%. Trace analysis after matched safety fine-tuning suggests that safety judgments are more likely to govern responses before harmful planning begins. A conditional theoretical analysis further characterizes when invariance learned from cunning data can transfer to safety-related inputs. These findings suggest that cunning data can strengthen model vigilance and complement conventional safety alignment.