aisecurity.ninja

Red Teaming Visual Language Models

Mukai Li (The University of Hong Kong), Lei Li (The University of Hong Kong), Yuwei Yin (The University of Hong Kong), Masood Ahmed (The University of Hong Kong), Zhenguang Liu (Zhejiang University), Qi Liu (The University of Hong Kong)

Published Jan 23, 2024 ยท arXiv:2401.12915

cs.AIcs.CLcs.CV

Abstract

The paper presents RTVLM, the first red teaming dataset to benchmark current Vision-Language Models (VLMs) in terms of faithfulness, privacy, safety, and fairness, revealing that prominent open-sourced VLMs struggle with red teaming in different degrees and have up to a 31% performance gap with GPT-4V.

arXiv abstract PDF