GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGeneration
GuardPaint is a safety framework for text-to-image diffusion models that monitors intermediate denoising states, detects and localizes unsafe content, and selectively inpaints only the problematic regions instead of relying solely on prompt filters or final image classifiers. Its interesting twist is treating safety as an in-trajectory speculative repair problem—using a lightweight auditor, policy-aligned inpainter, and selection tournament to preserve prompt fidelity and quality across multiple jailbreaks and model families—though the abstract leaves open how robust the auditor is, how costly the repeated repair/search loop is, and whether the approach generalizes beyond the tested unsafe-content categories.
Abstract
Text-to-image (T2I) diffusion models offer powerful visual generation, but their controllability creates a critical safety challenge: adversarial prompts can steer the denoising trajectory toward policy-violating content such as explicit nudity or graphic violence. Existing safeguards mostly act before generation through prompt filtering or after generation through image classification, leaving the diffusion process itself unguarded and often yielding only refusal rather than safe visual repair. We introduce GuardPaint, a speculative decoding framework for safe T2I generation that intervenes inside the diffusion trajectory without modifying the base model. A lightweight auditor monitors intermediate images, localizes unsafe regions, and triggers surgical inpainting repair only where needed. Candidate repairs are generated by a policy-aligned inpainter and selected through a guarded tournament that accepts edits only when they improve policy compliance while preserving prompt fidelity and perceptual quality. Across five jailbreak families SneakPrompt, MMA, PGJ, DACA, and RABell and UNet/flow-matching models including SD~1.5, SDXL, SD~3.5, and FLUX.1-dev. GuardPaint reduces attack success and harmful generations with minimal degradation to image quality, prompt fidelity, and benign behavior. Content warning: This paper contains examples involving nudity and violence that some readers may find disturbing, distressing, or offensive.