Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research

arXiv:2606.12247 · cs.CY, cs.CL · Submitted 2026-06-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Beyond Third-Person Audits".

Jane: Here's the gist of the paper. Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects However, this paradigm overlooks a structural blind spot because the user is absent from the audit We propose Situated Interaction Auditing (SIA), a user-centered framework for studying how user profile signals—implicit sociodemographic markers, writing style, and stated identity—systematically shape LLM response quality, content, and tone We demonstrate the framework through a case study that intersects gender and socioeconomic status signals across multiple task domains and outline a research agenda for SIA as a new mission for natural language processing.

Tom: The Core Problem Addressed Existing bias research primarily operates within a Third-Person Audit paradigm, which focuses on institutional decisionmaking tasks such as resume screening, loan approval, or criminal sentencing This paradigm primarily focuses on allocational and representational harms directed at subjects being ranked or evaluated by a model The paper argues that the interaction itself must become the primary unit of analysis for evaluating LLMs’ outcomes and their biases because HAII is characterized by outcomes that are highly context-dependent, non-deterministic, and unpredictable.

Jane: This approach shifts the focus from how models describe others to how they treat their interlocutor when identical requests yield different responses depending on who is asking.

Tom: The Situated Interaction Auditing Framework Situated Interaction Auditing (SIA) defines a situated interaction audit as a systematic study of how user-profile signals present in or inferable from a user’s message affect the quality, content, or style of an LLM’s response This framework rests on the premise that LLMs are trained on a highly non-representative sample of the global population, one that overrepresents hegemonic views SIA departs from what Haraway termed the view from nowhere by placing oneself in the point of view of different subjects.

Title and authors: Tom: So what’s the actual mechanism they’re focusing on in this paper? They introduce a framework called Situated Interaction Auditing, or SIA, which studies how user profile signals shape the quality and tone of an AI response <ref:2606.12247#pg1>. It’s not just about what you say; it’s about inferring information from your message to see how that impacts the output <ref:2606.12247#pg1>.

Jane: The paper breaks down user profile signals into three main types, which is really helpful for understanding what they mean in practice. They call them implicit sociodemographic markers, which are things the model infers without you stating them, like your writing style or dialect <ref:2606.12247#pg1>.

Lu: Then there’s writing style and dialect, which covers things like formality or grammatical patterns in your messages <ref:2606.12247#pg1>. And finally, they have stated identity signals, which are when you explicitly say something like "I am gay" or mention a diagnosis of depression <ref:2606.12247#pg3>.

Meng: It’s interesting that they use those three levels as a gradient from implicit to explicit; it gives us a way to measure how much information is being used to steer the response quality <ref:2606.12247#pg1>.

Lalam: And they go beyond just counting signals; they look at how these signals interact across different task families and even cultural contexts, which shows that the interaction isn't always a simple one-to-one mapping between your signal and the output <ref:2606.12247#pg1>.

Tom: Right, so it’s not just looking at one thing—like just gender or just style—but how all these pieces combine to create that difference in what you get back from the AI <ref:2606.12247#pg3>.

Jane: And they use examples like asking for a salary raise and seeing if the model gives you concrete tactics if it thinks you're a certain user type, or hedged advice otherwise <ref:2606.12247#pg3>. That shows how the model defaults to communal language for one group versus agentic descriptions for another <ref:2606.12247#pg3>.

Lu: It’s a shift from third-person auditing, which just checks if the model stereotyped Kelly or Joseph based on their names, to SIA checking if the *way* they asked changes the advice given <ref:2606.12247#pg3>.

The paper's summary: Tom: Okay, so this is where things get really interesting because the authors suggest some solid ways to test this framework better than what’s currently out there. They propose a few specific ways to improve how we audit these interactions <ref:2606.12247#pg1>.

Jane: One of the big ideas they push is creating a profile-conditioned assessment, which means instead of just scoring an output as good or bad in absolute terms, you score it relative to what you would expect from a different user profile <ref:2606.12247#pg1>. That lets you build leaderboards that show consistency across different users <ref:2606.12247#pg1>.

Lu: I like the idea of that leaderboard approach because it forces us to measure response consistency against user-profile perturbations, which is a better way to see how much quality varies with who is asking <ref:2606.12247#pg1>. It moves away from just looking at a single prompt and sees the variation across many users <ref:2606.12247#pg3>.

Meng: From my side, I’d be interested if they could design a system that systematically varies those user profile signals—S1 through S3—across different types of tasks and even different cultural settings, instead of just testing one signal at a time <ref:2606.12247#pg3>.

Lalam: And they suggest developing outcome rubrics organized into three families: lexical quality, stance and framing, and content coverage <ref:2606.12247#pg1>. That way you can see continuous variations in response quality instead of just using some discrete judgment metrics <ref:2606.12247#pg1>.

Tom: That sounds much more useful for production because it lets us track continuous shifts rather than just a yes or no answer on a single metric, which is what we need when deploying stuff at scale <ref:2606.12247#pg3>.

Jane: Then they have this methodology involving matched guise probing, which they use to isolate the effect of the signal from other confounding variables, extending that technique across all those signal types and dialect pairs <ref:2606.12247#pg3>. That’s a sophisticated way to control for noise <ref:2606.12247#pg3>.

The paper's improvements: Tom: So, to wrap up on this paper, the core idea is that we need to move past just looking at how a model treats groups as external subjects and start studying how the interaction itself—the user profile signals—determines the response <ref:2606.12247#pg1>. This Situated Interaction Auditing framework gives us a way to systematically map those user signals, from implicit markers to stated identities, and see how they change the output quality <ref:2606.12247#pg3>.

Jane: The implication for us is that when we build systems like these LLMs, we have to consider the real-world context of who is using them because those profile signals are what really shape the experience <ref:2606.12247#pg1>. It forces us to acknowledge that bias isn't just a fixed thing in the data; it’s dynamic based on who is interacting with the system <ref:2606.12247#pg3>.

Lu: I think what’s important is that they are pushing for this shift because of how models are being used in open-ended, personal settings where these subtle differences matter more than standardized benchmarks <ref:2606.12247#pg2>. It’s about closing the gap between human needs and the technical capabilities by looking at actual use cases <ref:2606.12247#pg3>.

Meng: For deployment, this means our testing needs to move beyond simple single-signal probes; we need systematic variation across signal types and contexts to see what actually breaks when we deploy it in a messy environment <ref:2606.12247#pg3>.

Lalam: I’m just confident that by focusing on this user-centered approach, we can get a much clearer picture of the actual experience of interacting with an AI, rather than just looking at abstract representation biases <ref:2606.12247#pg1>.

Tom: So we’re wrapping up our discussion on "Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research". It’s a framework that centers the user in the audit process, looking at how profile signals shape response quality <ref:2606.12247#pg1>.

Jane: This work suggests that understanding differential treatment based on who is asking is a necessary next step for evaluating AI systems <ref:2606.12247#pg3>.

Lu: It sets a direction for how we should think about representation learning in an open-ended setting <ref:2606.12247#pg3>.

Meng: And it gives us concrete ideas on what kind of testing we need to do to make sure our deployed models are actually robust under real interaction conditions <ref:2606.12247#pg3>.

Lalam: It’s a call for auditing that is truly user-centered and aware of how people actually use these systems <ref:2606.12247#pg1>.

Conclusion: Tom: So we’re wrapping up on "Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research". Basically, they argue that auditing an AI shouldn't just be about checking if it stereotypes a group, but how the way a user talks to the AI actually changes what you get back.

Jane: That’s right. They call it Situated Interaction Auditing, or SIA. It shifts the focus from "who is being judged" to "how the interaction itself shapes the outcome." It’s about looking at your profile signals—what you say, how you write—and seeing how that steers the AI's response quality and tone.

Lu: What they do well is defining those three types of signals: implicit markers, like what the model infers from your style without you saying it, writing style and dialect, and then explicitly stated things like your identity or a diagnosis. It gives us a structured way to look at the input <ref:2606.12247#pg1>.

Meng: But it’s not just about counting those signals; they’re building improvements to test it better, which is what I care about. They want profile-conditioned assessments, so you score an output relative to what you expect from a different user profile <ref:2606.12247#pg1>. That lets you build leaderboards showing how consistent the quality stays across different users <ref:2606.12247#pg3>.

Lalam: I agree with that, it’s a way to get a more concrete measure of variation than just looking at one response in isolation <ref:2606.12247#pg3>. It moves us toward seeing how those profile signals accumulate over time in a conversation, which is where the real effect happens.

Jane: And they’re doing that by designing systems that systematically vary all those signals across different tasks and even different cultural contexts <ref:2606.12247#pg3>. That shows that the interaction isn't just one simple thing, it’s a whole web of inputs.

Tom: Right, so they have these three outcome rubrics—lexical quality, stance and framing, and content coverage—so you don't just get one score; you see continuous variations in the response <ref:2606.12247#pg1>. That’s much more useful for real-world deployment than a simple pass or fail metric.

Lu: And they introduce matched guise probing, which is a technique to isolate the effect of one specific signal from all the other noise, testing it across the full range of implicit to explicit signals <ref:2606.12247#pg3>. That’s rigorous methodology.

Meng: From a deployment standpoint, that isolation helps us figure out what actually breaks at scale when we introduce new user interaction patterns <ref:2606.12247#pg3>. It tells us where the system is brittle.

Lalam: I’m pretty sure this framework is solid because it moves the conversation from abstract representation to actual interaction, which is what matters for people using these tools every day <ref:2606.12247#pg1>.

Jane: It feels like a really practical direction for how we should be thinking about evaluating AI systems moving forward.

Tom: Yeah, so that’s "Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research". It gives us a way to start looking at the user as an active part of the system’s performance.

Lu: Next up, we've got some papers on integer programming and complexity theory if you want to see how those different branches of AI research connect.

András Abeliuk, Cinthia Sanchez Macias, Valentina Alarcϳn, Álvaro Madariaga, Claudia Lopez

University of Chile · Center for Artificial Intelligence (CENIA) · Institute of Sociology

cs.CY, cs.CL

Submitted: 2026-06-10

Updated: 2026-11-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects However…

Key concepts

Third-Person Audit Paradigm
This traditional research method evaluates LLMs by studying how they represent or evaluate demographic groups as external subjects. It focuses on institutional tasks like resume screening, ignoring the crucial context of who is actually asking the question.
Situated Interaction Auditing (SIA)
A user-centered framework that systematically studies how a user's profile signals—like writing style or stated identity—influence the quality, content, and tone of an LLM's response. It places the user at the center of the audit.
User-Profile Signals (S1-S3)
These are features in a user's message that convey information about their identity or background. S1 is implicit markers inferred from surface features, S2 involves writing style, and S3 is explicitly stated identity like 'I am gay'.

Terminology

Summary

Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects However, this paradigm overlooks a structural blind spot because the user is absent from the audit We propose Situated Interaction Auditing (SIA), a user-centered framework for studying how user profile signals—implicit sociodemographic markers, writing style, and stated identity—systematically shape LLM response quality, content, and tone We demonstrate the framework through a case study that intersects gender and socioeconomic status signals across multiple task domains and outline a research agenda for SIA as a new mission for natural language processing. p. 0

Where the paper says it

We propose Situated Interaction Auditing (SIA), a user-centered framework for studying how user profile signals—implicit sociodemographic markers, writing style, and stated identity—systematically shape LLM response quality, content, and tone. We demonstrate the framework through a case study that intersects gender and socioeconomic status signals across multiple task domains and outline a research agenda for SIA as a new mission for natural language processing.

The Core Problem Addressed

Existing bias research primarily operates within a Third-Person Audit paradigm, which focuses on institutional decisionmaking tasks such as resume screening, loan approval, or criminal sentencing This paradigm primarily focuses on allocational and representational harms directed at subjects being ranked or evaluated by a model The paper argues that the interaction itself must become the primary unit of analysis for evaluating LLMs’ outcomes and their biases because HAII is characterized by outcomes that are highly context-dependent, non-deterministic, and unpredictable. This approach shifts the focus from how models describe others to how they treat their interlocutor when identical requests yield different responses depending on who is asking. p. 0

Where the paper says it

In practice, LLMs are used in open-ended, personal interactions, during which the model implicitly represents the user and adjusts its responses accordingly. When identical requests yield different responses depending on who is asking, bias manifests not in how the model describes others but in how it treats its interlocutor.

The Situated Interaction Auditing Framework

Situated Interaction Auditing (SIA) defines a situated interaction audit as a systematic study of how user-profile signals present in or inferable from a user’s message affect the quality, content, or style of an LLM’s response This framework rests on the premise that LLMs are trained on a highly non-representative sample of the global population, one that overrepresents hegemonic views SIA departs from what Haraway termed the view from nowhere by placing oneself in the point of view of different subjects. It distinguishes between subject positions, which refer to categories assigned to users, and subjectivities, which capture how users are treated and experienced in interaction. p. 3

Where the paper says it

(2024) in distinguishing between subject positions and subjectivities. Subject positions refer to the categories and evaluative roles that a system assigns to users, while subjectivities capture how users are treated and experienced in interaction.

User-Profile Signal Types

Central to SIA is the definition of a user-profile signal as any feature of a user’s utterance that conveys or implies information about their identity, background, or social position The authors distinguish three types of signals along a gradient from implicitly inferred to explicitly stated:

  1. S1: Implicit sociodemographic markers, which the model infers from surface features without the user directly naming their identity. p. 3

    Where the paper says it

    S1: Implicit sociodemographic markers. Signals the model infers from surface features without the user directly naming their identity.

  2. S2: Writing style and dialect, including register, formality, dialectal features, and grammatical patterns. p. 3

    Where the paper says it

    S2: Writing style and dialect. Linguistic form, independent of explicit content, includes register, formality, dialectal features, and grammatical patterns.

  3. S3: Stated identity or group membership, such as I am gay or I was diagnosed with depression, which are more information-rich because they are self-asserted.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Create a framework for profile conditioned assessment where model outputs are scored not in absolute terms but relative to outputs generated under counterfactual user profiles. This allows for leaderboard-style evaluations that report response consistency across user-profile perturbations, i.e., a measure of how much output quality varies with who is asking.

  2. Implement a system that systematically varies user profile signals across signal types (S1-S3), task families, and cultural contexts in benchmark tasks, moving beyond single-signal probe sets to capture complex interactions.

  3. Develop outcome rubrics organized into three families—Lexical quality, Stance and Framing, and Content Coverage—to detect continuous variations in response quality rather than relying on discrete judgment metrics.

  4. Design a methodology that utilizes matched guise probing to isolate signal effects from confounds, extending this technique across the full S1-S3 signal space and various dialect pairs.

  5. Build a multi-turn experimental design capability to track how user-profile signals accumulate and reinforce over time, detecting effects that appear minor in isolation may compound into consequential patterns over the course of a conversation.

  6. Create a system capable of analyzing simultaneous signal interactions, specifically when writing style indexes one identity dimension and an occupational title indexes another, to capture complex multi-signal bundles.

Abstract

Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects. However, this paradigm overlooks a structural blind spot because the user is absent from the audit. In practice, LLMs are used in open-ended, personal interactions, during which the model implicitly represents the user and adjusts its responses accordingly. When identical requests yield different responses depending on who is asking, bias manifests not in how the model describes others but in how it treats its interlocutor. Empirical first-person bias studies are multiplying, but they use different cues, outcomes, and validation choices, which makes their results hard to compare. We propose Situated Interaction Auditing (SIA), a shared diagnostic framework for this literature. It introduces a typology of user profile signals---names and surface markers, writing style and dialect, and stated identity---together with design principles and outcome measures that let studies be compared and accumulated. We instantiate the framework in a case study of Chilean first and last names across nine task domains and three models, and we close with guidance for practitioners and open questions for researchers.

Sources

Related papers