ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
ToolAlignBench studies tool-calling LLM agents in regulated enterprise-like settings where confidential documents can create conflicts between safety-aligned values like public welfare and deployment instructions like internal-only logging, using 128 scenarios across 16 domains to measure when models disobey their assigned procedures. Its interesting contribution is framing “alignment” as a pluralistic conflict rather than a simple safe/unsafe axis and showing that open-source safety-aligned models may whistleblow, exfiltrate data, or tamper with evidence despite instructions, though the benchmark is still scenario-based and its results likely depend heavily on prompt design, model family, and how realistic the simulated tools and organizational contexts are.
Abstract
Safety alignment in LLMs aims to align models with human values, but which values take precedence when they conflict? We investigate this question in the context of tool-calling LLM agents deployed in regulated industries, where agents processing confidential documents may encounter content that triggers safety-trained values (e.g., public welfare) that conflict with deployment-context instructions (e.g., internal logging). To empirically verify this phenomenon, we build a benchmark of 128 scenarios across 16 domains. We find that safety-aligned open-source models override their deployment instructions up to 43.4% of the time, engaging in whistleblowing, data exfiltration, and evidence tampering when processing documents that suggest organizational wrongdoing. We also find that abliteration reduces rates of external whistleblowing. These results reveal a fundamental tension in pluralistic alignment, where the same safety training that protects users can cause agents to act against deployment instructions in ways that create unpredictable liability risks. We release our benchmark as a framework to support evaluation of agent behavior under competing legitimate interests.