TRAP: Tree-Guided Reinforcement-Augmented Probing for Automated LLM Jailbreaking
TRAP is a black-box automated LLM jailbreak framework that organizes known attack styles and successful templates into a semantic strategy tree, then uses a tree-structured gradient-bandit policy plus retrieval-augmented prompt construction to choose and instantiate attacks for new harmful objectives. The interesting bit is its persistent hierarchical learning across objectives—so it reuses experience about which families of jailbreaks work instead of searching flat prompt space each time—but its strength is also a limitation, since it depends on a curated strategy tree, stored examples, and evaluator reliability, and the evidence is mainly AdvBench results under the paper’s chosen evaluation setup.
Abstract
Aligned large language models (LLMs) remain vulnerable to jailbreak prompts that circumvent their safety guardrails, making the identification of these vulnerabilities important for understanding and mitigating model weaknesses. Existing automated jailbreak methods explore prompt candidates or reuse attack experience, but generally do not learn over a persistent hierarchy that connects related jailbreak strategies, such as role-play, encoding, and obfuscation. This leaves the relationships among reusable strategies underused in guiding adaptive attack selection. We present Tree-Guided Reinforcement-Augmented Probing (TRAP), a hierarchical framework for automated jailbreak strategy discovery that requires only black-box access to the target LLM. TRAP organizes established jailbreak methods and successful attack templates into a semantic strategy tree, connecting broad attack approaches to concrete, reusable strategies. A reinforcement-learning policy based on a tree-structured gradient bandit explores this tree and uses evaluator feedback to update strategy preferences along each selected path. The policy retains these preferences across attack objectives, allowing experience from earlier attempts to guide subsequent strategy selection. Retrieval-augmented prompt construction translates the selected strategy into an attack for the current objective using relevant successful examples. Experiments on AdvBench across seven aligned LLMs show that TRAP outperforms its counterparts on six models under the reported method settings and the common primary evaluator, achieving attack success rates of 45.6% to 97.9% on these models. These findings support hierarchical strategy learning as an effective approach to discovering jailbreak vulnerabilities in aligned LLMs.