Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access
The paper frames “prompt privilege” as an accessibility problem where LLMs give different-quality answers to semantically equivalent requests depending on users’ literacy or prompt-engineering skill, and proposes a Prompt Equity Score to measure these disparities plus a Prompt Equity Transformer that rewrites user inputs into more effective intent-preserving prompts. Its nice contribution is shifting prompt optimization from users to an automated accessibility layer and showing on MedQA that this can reduce measured disparities, though the evidence seems narrow—mostly one benchmark/domain and dependent on the transformer’s ability to preserve intent reliably across real-world users and tasks.
Abstract
Large language models (LLMs) are increasingly integrated into healthcare, education, public services, and everyday decision making. They should provide comparable assistance regardless of a user's literacy, communication style, or prompt-engineering expertise. However, existing research on prompt robustness primarily focuses on adversarial attacks, prompt injection, and prompt optimization, while overlooking whether semantically equivalent requests receive different responses simply because they are phrased differently. We refer to this accessibility challenge as "Prompt Privilege": users with greater prompting expertise systematically obtain better model performance despite expressing the same underlying intent. To address this problem, we present a unified framework for measuring and mitigating accessibility disparities in LLM interactions. We introduce Prompt Equity Score (PES), a quantitative metric for evaluating performance consistency across user populations, and Prompt Equity Transformer (PET), an LLM-based agent that automatically transforms user requests into semantically equivalent, accessibility-oriented prompts while preserving their intent. PET shifts prompt optimization from the user to the AI system, functioning as an intelligent accessibility layer between users and foundation models. Experiments on the MedQA benchmark demonstrate measurable prompt privilege, with statistically significant performance disparities between low-literacy and expert-prompting cohorts. Applying PET eliminates these disparities while preserving semantic fidelity, demonstrating that accessibility-oriented prompt normalization can improve equitable AI access. By introducing prompt privilege as a new dimension of AI accessibility and PET as a practical solution, this work advances system-centered accessibility and provides a foundation for more fair, trustworthy, and inclusive AI systems.