BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views
BrowseSafe++ is a large paired benchmark for evaluating prompt-injection detection in browser agents, covering nearly 39k attacked/clean webpage pairs across HTML, Markdown, accessibility-tree, and screenshot views, with real-world injections embedded in privacy-sanitized page contexts and associated user tasks. What’s neat is that it treats prompt injection as a web-context problem rather than just a text-classification problem, adds cheap small-model guards that outperform frontier-model-based detection, and shows adaptive attacks can jump from low success to over 95%, though the privacy rewrites and benchmark-specific training may limit how cleanly the results transfer to fully live web settings.
Abstract
Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect legitimate tasks. We introduce BrowseSafe++, which extends BrowseSafe from 14,719 pages to 38,675 pairs of pages. Most of its prompt injections were observed in the wild on production websites. We keep each injection unchanged and rewrite the sensitive parts of the page around it to protect privacy. Each attacked page has a clean copy of the same page as its control. Both are provided as HTML, Markdown, an accessibility tree, and a screenshot. Every pair includes a user task and its expected answer. The benchmark includes twelve types of websites, from news articles to shared documents and forums, with 24 attack types appearing in 18 different locations on a page. We use these paired pages to train BrowseSafe-Guard, two 0.8B-parameter detectors for text and screenshots. Both achieve higher F1 scores than the detector based on frontier models, while keeping false-positive rates on benign pages and inference cost low. BrowseSafe-Adapt revises benchmark injections from browser-agent feedback, and attacks that succeed on only a few percent of tasks when replayed directly reach up to 95.41% success once adapted.