Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data
The paper tackles prompt-injection defense for LLMs using external documents or tool outputs by reframing the problem as per-instruction classification: identify each embedded directive, decide whether it is beneficial or harmful, and train the model to comply selectively rather than blanket-trust or blanket-refuse external instructions. Its main novelty is EDIC, a decomposed RL reward pipeline with synthetic mixed-instruction data and a GRPO-refined generative judge that assigns instruction-level credit, improving both rejection and utility in experiments, though its effectiveness likely depends on the quality and coverage of the generated scenarios and judge, and the evaluation is still limited to a small set of model backbones and benchmarks.
Abstract
LLMs over external data face a per-instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Existing defenses suppress or trust all external instructions, collapsing the safety-utility frontier and forcing a choice between over-refusal and over-compliance. We argue the missing capability is \textbf{per-instruction credit attribution} and formalize it as External-Data Instruction Classification, an identify-classify-act decision over each embedded directive. We propose \textbf{EDIC}, a decomposed reward framework for per-instruction compliance. A scene-grounded pipeline injects mixed beneficial and harmful instructions with counterfactual filtering, while a generative judge is initialized by SFT and refined with GRPO to attribute compliance at the instruction level. The judge then produces a reward that separately encourages beneficial compliance and penalizes harmful compliance for RL training. Across two backbones and five benchmarks, EDIC improves both safety and utility over the strongest prior defense, lifting harmful-rejection by up to 7.7\% and Arena-Hard-Auto utility by 12.5\% on Qwen3-4B. The code repository is hosted at https://github.com/yingyingxia666/EDIC.