From Expected Harmfulness to Likelihood: A Probabilistic Reformulation of Jailbreaking LLM Agents
The paper reframes LLM-agent jailbreaking by showing that maximizing expected harmfulness of outputs and optimizing the likelihood of harmful target outputs are mathematically linked through a gradient identity involving a harmfulness-reweighted output distribution. Its neat contribution is turning target-output attacks into a principled sampling-and-optimization method, OPUR, rather than a heuristic target-picking trick, though the setup seems to rely on having a meaningful harmfulness scoring function and enough access to estimate or use these gradients effectively.
Abstract
When the harmfulness of an LLM agent's output can be quantified, a natural jailbreaking objective is to maximize expected harmfulness over admissible input modifications. An alternative approach constructs or selects harmful target outputs and modifies the input to increase their likelihood. We establish a precise connection between these two approaches through a probabilistic reformulation. Specifically, we show that the gradient of the logarithm of expected harmfulness with respect to the input equals the expected input gradient of the model's log-likelihood under a harmfulness reweighted output distribution. This identity provides a unified interpretation of expected harmfulness and target likelihood optimization. Building on this connection, we propose OPUR, a sampling distribution designed to generate highly harmful target outputs and use the resulting samples to guide likelihood-based input optimization. Experiments demonstrate the effectiveness of the resulting method in jailbreaking LLM agents.