A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination
The paper proposes HACA, an automated red-teaming framework for multimodal LLMs that decomposes image-text jailbreaks into atomic structural, semantic, and syntactic strategies, then uses a cross-modal planner and generator to combine them into executable attacks. Its main contribution is turning ad hoc multimodal jailbreak crafting into a systematic strategy space plus automated attack pipeline with very high reported success across five MLLMs, though the abstract leaves unclear how robust the evaluation is against adaptive defenses, how “success” is judged, and whether the generated attacks generalize beyond the tested models and benchmarks.
Abstract
Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges: the lack of a atomic multi-modal strategy space, the absence of a concise and efficient executable framework beyond human-craft experience. To address these challenges, we first decompose the text-image jailbreak strategy space into three levels: structural, semantic, and syntactic, constructing a jailbreak strategy set encompassing both text and image modalities to systematically achieve combined coverage of different attack types. Then we propose a multimodal automated red team jailbreak method named Hierarchical Atomic Combination Attack (HACA). Specifically, based on a six-dimensional strategy space, a cross-modal joint planner is used to select and combine the different atomic jailbreak strategy for subsequent jailbreak command generation. Finally, at the implementation level, we explore to apply a unified generate executor to directly generate jailbreak instructions based on the selected multi-modal strategies. A series of experiments show that our automated red team method can achieve an attack success rate of average 95.48\% against five mainstream MLLMs.