StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge
StableJudge is a lightweight defense framework for LLM-as-a-judge systems that detects and sanitizes score-hijacking prompt injections in evaluated content before passing it to the judge, using a small 0.13B “Sentry” model plus deterministic scoring from the judge’s output distribution. Its key idea is to decouple adversarial robustness from the judge model itself, making the defense reusable across judges and updates; the main caveat is that the evidence is abstract-level and benchmark-based, so real-world robustness will depend on how well the Sentry generalizes to messier domains, adaptive attackers, and non-textbook injection styles.
Abstract
Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific model's scale and training and requiring repeated effort when the judge is updated or replaced. We propose StableJudge, a lightweight defense framework that externalizes prompt-injection mitigation as a reusable pre-scoring capability. StableJudge centers on a compact 0.13B Sentry that localizes injected spans and performs targeted sanitization prior to evaluation, and is trained using adversarially evolving attacks to improve robustness beyond a fixed prompt set. To stabilize measurement, StableJudge derives deterministic, fine-grained scalar scores from the judge's output distribution. Across held-out scientific abstracts and out-of-distribution attack variants, StableJudge improves score recovery under attack while largely preserving general-purpose performance, demonstrating that robust LLM-based judging can be achieved by decoupling safety from judge scale via reusable defense capabilities. Code and data will be released.