Beware EviLLM: Enabling Vulnerability Injection via Large Language Models
The paper introduces EviLLM, a threat model where a third-party attacker compromises parts of the LLM-based code generation workflow—such as an account or browser session—to cause generated code to contain vulnerabilities across multiple CWE classes. Its main contribution is shifting the focus from accidental insecure code, malicious models, or malicious users to a more realistic pipeline-compromise attacker and backing it with prototype attacks plus a small user study, though the evidence seems limited by the study scale and by evaluating only two concrete attack instances.
Abstract
Advances in large language models (LLMs) have enabled AI-driven code generation from natural language specifications, introducing new attack surfaces for injecting vulnerabilities into software. Prior work has studied this problem only in benign settings where vulnerabilities are introduced inadvertently, or under unconventional threat models where the LLM itself is malicious (backdooring) or the user is the attacker (jailbreaking). In this paper, we study a more realistic threat model: a third-party adversary, with capabilities comparable to existing cybercriminals, compromises the AI code generation pipeline to deliberately introduce vulnerabilities. We call this the EviLLM attack. We have implemented two instances of EviLLM, each of which only requires the underlying LLM to be accessed through a compromised account or browser, and can inject vulnerabilities from 13 CWE classes. As we show in our feasibility study, both attack vectors are already used to implement many existing cyberattacks. Our user study shows that 7 out of 8 and 10 out of 13 participants did not notice the vulnerabilities injected by the two instances of EviLLM, and 13 out of 21 participants "rarely" or "never" considered the risk of an attack like EviLLM.