TACS: Trajectory-Aware Candidate Selection for LLM Jailbreak Suffix Optimization
The paper argues that gradient-based LLM jailbreak suffix optimization is bottlenecked not just by how candidate suffixes are generated, but by a myopic selection step that picks the lowest-current-loss candidate even when that choice often leads to worse attack outcomes later. Its contribution is TACS, a trajectory-aware selection method using future-looking proxies plus reference-policy regularization and a discriminator-estimated chi-squared correction, which reportedly improves HarmBench attack success and stability under the same budget, though the evidence seems limited to suffix-based jailbreak optimization and the abstract leaves open how well the extra machinery generalizes across models, defenses, and threat settings.
Abstract
Gradient-based jailbreak suffix optimization methods typically update the suffix by retaining the candidate with the lowest current loss. We show that this seemingly natural design is fundamentally myopic: candidates that look better under the current-step proxy often fail to produce better jailbreak outcomes later in the search, revealing a form of selection-stage reward hacking. This suggests that candidate selection, rather than candidate generation alone, is a hidden bottleneck in suffix optimization. To address this issue, we propose \OURS{}, a trajectory-aware candidate selection framework for jailbreak suffix optimization. Instead of selecting candidates solely by their immediate loss, \OURS{} augments per-step evaluation with a trajectory-aware proxy and stabilizes selection with reference-policy regularization and a discriminator-estimated chi-squared correction, encouraging choices that remain effective beyond the current step. Experiments on HarmBench show that \OURS{} consistently outperforms strong baselines under the same search budget, substantially improving attack success rates while exhibiting more stable optimization behavior throughout the search. Our findings highlight that mitigating selection-stage reward hacking caused by myopic candidate selection is critical for improving jailbreak suffix optimization.