TrustMI: Causally controlling how assistants trust their users
TrustMI studies whether an LLM assistant’s tendency to trust a user or third party can be treated as a controllable behavioral variable, building contrastive conversations around ability, benevolence, and integrity and using frozen-model activation steering to push trust decisions up or down across multiple instruction-tuned model families. The interesting bit is that “trust” appears to move monotonically along learned linear activation directions and transfers to safety-relevant agent scenarios like harmful requests, prompt injection, and insider threats, though the evidence is still limited by synthetic contrastive data, steering-style interventions, and an abstract notion of trust that may not fully capture real deployed agent behavior.
Abstract
Large Language Model (LLM) assistants routinely decide whether they can trust users and third parties whose competence, intentions, and integrity they cannot verify. This uncertainty matters for safety, as trusting the wrong party can lead an agent to comply with harmful requests or act on malicious instructions encountered during tool use. To study this problem, we define trust as an assistant's willingness to accept vulnerability to the actions of another party and ask whether such behavior can be causally controlled through model activations. We build 2,000 contrastive conversations spanning ability, benevolence, and integrity, where paired responses complete the same request but differ in whether the assistant trusts the user. From these pairs, we learn steering matrices while keeping the model parameters frozen and test them across six instruction-tuned models from three families, finding that steering changes trust decisions monotonically in both directions. We then ask whether this effect extends to several safety-related agent settings involving harmful requests, prompt injections, and insider threats, while using benign-task and reasoning as controls. Our findings provide evidence that trust in the user can be causally controlled along linear directions in model activations and provide a way to study how trust shapes safety-relevant behavior in language models.