Jailbreaking is Best Solved by Definition

Abstract

This work critically examines the two stages of the defense pipeline against "jailbreak" attacks on language models, the definition of unsafe outputs and the enforcement of the definition, and casts doubt on the efficacy of existing enforcement mechanisms by showing that they fail to defend even for a simple definition of unsafe outputs.