EvolvAttack: Backdooring self Evolving VLM Agents
EvolvAttack studies a backdoor threat for self-evolving VLM agents where the malicious behavior is not present in the initial static agent, but becomes active only after the agent’s memory has evolved and a visual trigger appears. The interesting bit is that the attack binds trigger activation to memory state, so ordinary static backdoor scans with clean memory miss it, while experiments across several VLM backbones, agent pipelines, and memory-update schemes show high post-evolution attack success with little pre-evolution activity; the main caveat is that the evidence comes from the paper’s tested agent/memory setups, so real-world robustness and defenses remain open questions.
Abstract
Self-evolving mechanisms let vision-language model (VLM) agents continually adapt through evolving memory, but they also allow malicious behavior to emerge only after deployment. We introduce a new attack surface, the self-evolving backdoor, whose payload stays dormant in the initial agent and is progressively activated as the agent evolves. Unlike conventional backdoors, which can be exposed by testing triggered inputs on a static model, our attack requires both a visual trigger and a sufficiently evolved memory state; static scanning with clean memory thus yields near-zero attack success and may deem the agent benign. We realize this through memory-aware backdoor training, which couples trigger activation with the agent's evolving memory while preserving benign behavior unless the trigger co-occurs with payload-related memory. Across three VLM backbones, four agent pipelines, and recency-, semantic-, and reward-based memory evolution, the attack preserves normal functionality and stays nearly inactive before evolution (0.5% attack success rate), yet exceeds 95% attack success within 100 interactions, revealing a stealthy threat unique to self-evolving agents.