AKRASIA: Stealthy Backdoor Attack on Reasoning-based Code LLMs
AKRASIA studies an inference-time backdoor attack on reasoning-oriented code LLMs, where an attacker probes the model to craft code-level triggers, uses in-context examples to teach the backdoor behavior, and exploits unfaithful chain-of-thought-style reasoning to hide the trigger while still producing plausible explanations. What stands out is that it targets the reasoning layer rather than just the model weights or training data, showing very high attack success while largely preserving task accuracy and bypassing both automated defenses and human review, though the abstract leaves open how realistic the probing/setup assumptions are and how well the results generalize beyond the tested models, tasks, and defenses.
Abstract
We present AKRASIA, a stealthy, inference-time backdoor attack against reasoning-based Code LLMs. AKRASIA aims to achieve a backdoor target (e.g., malicious code execution) in reasoning LLMs while evading automated defenses and human inspection. To achieve this, AKRASIA probes the victim LLM to construct a code-level backdoor trigger. It then employs in-context learning for backdoor learning, and model unfaithfulness to conceal the backdoor trigger, and generate plausible reasoning. We evaluate AKRASIA using four backdoor targets six (6) reasoning LLMs, three coding tasks/datasets and three defense methods. AKRASIA has up to 99.34% average attack success rate on SOTA LLMs and mantains up to 97.23% average accuracy. AKRASIA evades the SOTA defense, retaining up to 98.82% average ASR in most (14/18) defense settings. It evades human inspection, successfully hiding the backdoor trigger and reasoning steps in up to 80% of settings. Our findings motivate the need to defend LLMs against reasoning backdoors.