Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
This survey maps the safety landscape of multimodal large language models, focusing on how modality alignment, fusion, and cross-modal reasoning introduce threats beyond text-only LLMs, including adversarial inputs, poisoning, jailbreaks, hallucinations, modality misalignment, and fused safety failures. Its useful angle is a multimodal-grounded taxonomy that reframes threat models and defenses around cross-modal interactions rather than porting over unimodal assumptions, though as a survey it appears more conceptual and organizing than empirical, so its value depends on the completeness and clarity of the taxonomy rather than new experimental results.
Abstract
Multi-modal large language models (MLLMs) integrate heterogeneous modalities through modality alignment and fusion, enabling stronger understanding and reasoning. However, this architectural shift reshapes the safety landscape of machine learning. Increased model complexity and cross-modal interactions give rise to novel threats, including compromised modality integration, modality misalignment, and fused safety risks, reflecting shifts in threat modeling beyond uni-modal assumptions. These shifts, in turn, impose new constraints on safety solutions not captured by existing frameworks rooted in uni-modal learning. Motivated by these challenges, this survey provides a systematic analysis of the evolving safety landscape of MLLMs. We first propose a multimodal grounded taxonomy of safety threats and analyze shifts in threat models, covering adversarial attacks, data poisoning, jailbreaks, and hallucinations. We then summarize updated safety assumptions and organize recent advances in MLLM safety strategies accordingly. Finally, we discuss open challenges and future directions to inform the development of more principled and scalable safety mechanisms for multimodal systems.