From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference

arXiv:2601.11817 · cs.CY, cs.HC · Submitted 2026-01-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "From Defense to Advocacy".

Title and authors: Tom: So we’re looking at this paper today called "From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference". It’s about moving away from how we currently try to defend privacy and toward actively using what AI sees.

Jane: That sounds like a big shift because right now, most privacy rules feel like a shield you have to hold up yourself, constantly choosing between giving data access or losing it entirely. This paper seems to be looking at that exact problem of binary defenses.

Tom: Exactly. The authors argue that this choice between accepting data collection and refusing it isn't working anymore because AI is getting so good at making inferences about us without us ever knowing it, which they call the Blind Self.

Lu: From what I see in the paper, they are pointing out that current privacy policies focus too much on collecting data directly, but they ignore these hidden inferences that AI can make about you. It shifts the focus from just data collection to how we are actually being modeled by systems.

Meng: But how does this help someone who is just trying to use an app or a service? If it’s about "advocacy," what does that mean in practice for my daily life instead of just reading legal jargon?

Jane: Well, the core idea is introducing something called Personal Advocacy Agents, or PAAs. They think this agent can bridge the gap between you and these big organizations by revealing and interpreting those Blind Self inferences.

Tom: So imagine an AI agent that doesn't just tell you what it knows about you, but actually shows you the hidden stuff it inferred—the Blind Self—and then helps you decide how to react to that information.

Lu: They suggest this moves privacy management from being a passive defense mechanism to something active, where the user can strategically leverage or suppress what the AI has inferred about them. It’s about turning inference into something you can work with.

Meng: That sounds useful for managing risk, but if the AI agent is doing all that analysis, how do we know that the agent itself isn't introducing new biases or just making things more complicated?

Title and authors: Lalam: I think what they are proposing is a way to stop asking "Do you agree to share X?" and start asking questions like, "Do you want to be modeled as Y based on X?" which directly addresses those limitations in consent mechanisms.

Jane: That's a really concrete difference. It changes the whole conversation from a simple yes or no decision about sharing data to a more nuanced dialogue about how you want to be understood by an AI system.

Tom: They even talk about how this agent could perform an inference audit, using on-device AI to look at your browsing history or purchase records in real time and simulate what external systems would infer.

Meng: Real time auditing sounds powerful for security, but running that kind of analysis locally on my device has to be incredibly efficient. What’s the computational cost there?

Lu: They are aiming to approximate contextual reasoning by simulating how different pieces of information flow through different contexts, which gives users a computational framework to see if they are being modeled in a way that violates their norms.

Jane: So it’s not just about stopping data collection anymore, but understanding the trajectory—how you might be understood over time—and having tools to manage those paths.

Tom: The paper clearly shows that these binary defenses, the opt-in or opt-out model, are structurally flawed across governments and organizations because they don't account for how AI makes inferences about us in this Blind Self.

Meng: It sounds like the main critique is that current laws aren't even designed to protect what you *don't* know about yourself yet, which is a huge gap when AI is already doing that stuff.

Lu: Right, the structural deficiency they highlight is that privacy management relies too heavily on our own limited ability to process information and choose what to disclose.

Jane: So the shift they want us to make is moving from system transparency—which usually means looking at how a system operates—to self-transparency, which means looking at how an individual is being modeled.

Tom: And they focus on inference trajectories over time instead of just single disclosures, asking users what kind of model they want to be based on their past actions.

Meng: If we had these PAAs, could it help with things like job screening or personal branding where selective disclosure is key? The paper mentioned that as an example.

Title and authors: Lalam: Yes, the authors suggest that a user could use the agent to get advice like selectively disclosing information in specific contexts to enhance their personal brand, or conversely, using data obfuscation to suppress certain flows of information.

Jane: It’s about giving users agency back in how they present themselves when interacting with AI-driven systems. It gives them a way to proactively steer the inference process instead of just reacting after the fact.

Tom: So we’re moving from a defensive posture, where we are constantly trying to block things, to an advocacy posture, where we actively manage what kind of model is being built about us by AI.

Lu: The authors argue this isn't just a nice idea; it's a necessary response because the rapid growth of AI inference capabilities means this blind spot is becoming too dangerous to ignore.

Meng: I still worry about the actual implementation cost and ensuring these agents don't introduce new, unforeseen vulnerabilities into our personal data flow.

Jane: That’s a fair concern, but they seem very focused on making the process transparent so you can vet what you're agreeing to before it happens.

Tom: Overall, the main point of "From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference" is that we need a new way for people to manage privacy that isn't just about checking boxes anymore.

Lu: It’s about redefining privacy as epistemic power—the power you have over how you are represented by algorithms, not just controlling who gets access to the data itself.

Meng: So, instead of fighting every single data request, we use these agents to strategically manage the inferences that happen behind the scenes.

Jane: Exactly. It’s about giving users tools to understand and shape their own digital identity in an AI-driven world.

Tom: We’ll keep an eye on how this concept of Personal Advocacy Agents plays out when they get built into actual products.

Lu: And I think the way they frame inference trajectories over time is a really strong conceptual tool for future research in representation learning.

Meng: It’s a lot to digest, but if it means we can actually control the narrative of what AI knows about us, then maybe it’s worth exploring further.

The paper's summary: Tom: So, we’ve talked about how current privacy rules force us into this binary choice: either you give up your data access or you lose access entirely. This paper argues that this whole defense strategy is broken when AI starts making inferences about you that aren't obvious to you.

Jane: Right, and the core idea they are pushing is moving away from being a passive shield to an active advocacy role. They call it the personal advocacy agent or PAA concept.

Tom: Exactly. Instead of just trying to block data collection, this framework suggests we should use AI tools to actually see what the AI is inferring about us—that hidden Blind Self—and then figure out how to work with that information strategically.

Lu: What this means structurally is they’re redefining privacy as epistemic power instead of just access control. It shifts the focus from checking if a system is transparent, to figuring out how well an individual is being modeled by that system.

Jane: They introduce these agents to reduce the gap between you and the organization by revealing and interpreting those inferences. I like that they are making it about self-transparency rather than just system transparency.

Tom: They actually show how this agent can perform an inference audit using on-device AI, looking at things like your browsing history or purchase records in real time to simulate what external systems would infer from you. That’s a big shift toward proactive management.

Lu: They are proposing that users can get advice on how to strategically engage with those AI outputs. For example, they might suggest selectively disclosing information in a job screening context to enhance your personal brand, or conversely, advising you to use data obfuscation or selective deletion to suppress certain flows of information.

Jane: That moves the conversation from just agreeing to share something—that consent thing—to asking a much more nuanced question like, "Do you want to be modeled as Y based on X?" which seems like it fixes some of those consent mechanism problems.

Tom: And they get really technical about it by approximating contextual reasoning. They simulate how different pieces of information propagate through various contexts, giving users a computational framework to assess if they are being modeled in a way that violates their personal norms.

Lu: The paper is really focused on the "inference trajectories" over time, which is much more useful than just looking at one single data point or disclosure. It’s about understanding the pattern of how you might be understood by an AI system as you move through different parts of your life.

Jane: So, for someone listening who doesn't read technical papers, this means instead of constantly fighting every piece of data collection, we’d have a tool that helps us manage the story our own digital identity is being told.

Tom: It’s about turning the blind spot into something you can actually leverage or suppress, rather than just being trapped by it. The authors are pointing out that this isn't just a nice idea; it’s an active way to restrain how much of ourselves AI can learn without our knowledge.

Lu: They stress that this isn't a simple fix for everything either. They acknowledge the limitations of existing frameworks, which means they aren't claiming this agent solves every single problem on its own.

Jane: And what they don't do is solve the whole issue of building these agents in a way that doesn't introduce new biases or complexities into our personal data flow. It’s a big challenge ahead.

Tom: Right, so it’s about gaining agency over how we are represented by algorithms, and figuring out the practical cost and safety of doing that at scale.

The paper's improvements: Tom: So, we’ve been talking about the basic framework for personal advocacy agents and how they can audit your data flow in real time. Now, let's look at what improvements they suggest to make these agents actually useful for people on the ground.

Jane: Right, so if it’s about auditing what’s happening behind the scenes, what are the specific tools or advice they recommend users get from these agents?

Tom: They suggest giving users concrete advice. For instance, instead of just seeing a raw inference, an agent could advise you to selectively disclose that information in a job screening context to boost your personal brand. That’s about turning technical analysis into practical strategy.

Lu: And they also focus heavily on managing those inference trajectories over time by asking users specific questions like, "Do you want to be modeled as Y based on X?" instead of just the standard, vague consent question about sharing data.

Jane: That sounds like a direct fix for the limitations of existing consent mechanisms. It shifts the user's role from simply agreeing to a condition to actively steering their own representation in an AI system.

Tom: They also propose simulating how different pieces of information propagate through various contexts, which gives users a computational way to see if they are being modeled in a way that violates their personal norms, like seeing if your data is leaking into an area it shouldn't be.

Meng: That contextual reasoning part sounds ambitious. But for me, the question is about deployment and cost—how do we actually run these simulations on a user’s device without making things slow or too complex?

Lu: The paper acknowledges that approximating true contextual reasoning is hard, and they are focused on providing a computational framework that gives users actionable insights rather than claiming perfect accuracy.

Jane: So, the implication here is that we move toward giving people tools to manage their digital identity proactively instead of just reacting after something has already happened to them. It’s about taking control back from the opaque models.

Tom: It really boils down to shifting from a defensive posture, where you're constantly trying to block things, to an advocacy posture where you actively manage what kind of model is being built about you by AI.

Lu: The authors are essentially showing how this agent can help users strategically leverage the Blind Self and mine it for value instead of just letting it grow unchecked.

Jane: It’s a move toward making privacy management something that is personalized, contextual, and proactive rather than a one-size-fits-all defense mechanism.

Tom: We'll keep an eye on how these practical applications play out when they get built into actual products and see if we can get those real performance numbers.

Conclusion: Tom: So, to wrap up this discussion on "From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference," we've seen how this paper moves privacy management from a defensive shield to an active advocacy role using personal advocacy agents.

Jane: It’s really about giving users tools—these agents—to see the hidden inferences AI makes about them and decide how to react strategically, rather than just choosing between giving data or not.

Tom: Exactly. The core implication is that we need a way for individuals to understand *how* they are being modeled by AI, not just what data they are handing over directly.

Lu: This redefines privacy as epistemic power, meaning the ability to shape how you are represented rather than just controlling access to the service. It’s about moving from system transparency to self-transparency.

Meng: I still have my deployment concerns, though—if these agents are doing all this complex contextual reasoning, what is the computational cost of running that simulation on a user’s actual device at scale?

Lalam: From my end, I’m confident in the direction they suggest. My analysis shows that this approach to inference trajectories provides a very strong mechanism for users to manage their exposure over time.

Jane: So, we move from asking "Do you agree to share X?" to asking questions about modeling—like "Do you want to be modeled as Y based on X?" which seems like a much better way for people to engage with AI systems.

Tom: It's a big step because it addresses the structural flaw in current consent models that ignores these hidden inferences. It’s about giving users agency back into the process.

Lu: The paper shows how this agent can approximate contextual reasoning by simulating how different pieces of information flow through various contexts, which gives people a framework to check for normative violations.

Meng: That framework is useful theoretically, but until we see that running these simulations is efficient enough for real traffic, I'm skeptical about the practical deployment feasibility.

Jane: Well, it certainly sets a direction for future research on how to make this concept of personal advocacy agents actually work in the real world.

Tom: It definitely shows we need to stop focusing just on system security and start focusing on how individuals are being represented by those systems. That’s what "From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference" is about.

Lu: It’s a necessary shift because the Blind Self isn't just a data point; it's the new frontier for privacy risk, and this paper offers a way to confront it.

Lalam: I’m sure this direction will be taken seriously by developers who need to build safer, more transparent AI interactions.

Jane: Next time we look at a paper, we’ll see how these agents actually perform those inference audits in practice.

YUMOU WEI, JOHN CARNEY, JOHN STAMPER, NANCY BELMONT

Carnegie Mellon University · MARi LLC

cs.CY, cs.HC

Submitted: 2026-01-16

Updated: 2026-11-18

Comments: 13 pages, 2 figures; v2: substantially revised

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: Most privacy regulations function as a passive defensive shield that users must wield themselves The rapid expansion of Blind Self—attributes known to algorithms but unknown to the user—emerges…

Key concepts

Blind Self
These are attributes about you that algorithms know but you are unaware of. They represent the hidden inferences an AI makes about your behavior, which can be both revealing and exploitable by external systems.
Personal Advocacy Agents (PAAs)
Conceptual design patterns that reduce the gap between users and organizations. PAAs reveal and interpret Blind Self inferences, making the AI's logic transparent so users can understand how they are being modeled.
Inference Trajectories
This framework shifts privacy focus from single data points to a continuous path of modeling over time. It allows users to ask strategic questions like, 'Do you want to be modeled as Y based on X?' instead of just agreeing to share one piece of data.
Epistemic Power
Privacy is redefined not as simple access control but as the power an individual has over knowledge. This means focusing on how you are being understood (modeled) rather than just controlling what data you share.

Terminology

Summary

Most privacy regulations function as a passive defensive shield that users must wield themselves The rapid expansion of Blind Self—attributes known to algorithms but unknown to the user—emerges as a critical challenge We propose a paradigm shift from defensive privacy management to proactive privacy advocacy By illuminating the hidden inferences that users can strategically leverage or suppress, these agents not only restrain the growth of the blind self but also mine it for value

The Problem with Binary Defenses

Most current privacy regulations rely on a fundamental defense mechanism and uses what we call “binary defenses” to protect privacy Under this mechanism, users are effectively forced to choose between two extremes: either accept data collection and gain access to the service, or refuse data collection and typically lose access This binary choice simplifies the complex and ongoing trust negotiation process [24] These defenses are increasingly inadequate in contemporary data processing environments, especially given the proliferation of AI systems that use sophisticated algorithms for accurate inferences [2601.11817>

Structural Deficiencies Across Paradigms

The paper analyzes structural deficiencies across governmental, organizational, and institutional levels regarding binary defenses At the governmental level, privacy self-management relies too heavily on users’ bounded cognitive and information processing abilities In the organizational paradigm, security is often viewed as a proxy for privacy rather than a substantive question of power or trust At the institutional level, the “accuracy trap” occurs where individuals have the right to correct facts but not to challenge interpretations derived from them

The Personal Advocacy Agent Solution

The authors propose a new privacy management paradigm called “personal advocacy” which views privacy as contextualized agency They introduce the concept of “Personal Advocacy Agents” (PAA) as a conceptual design pattern to reduce epistemic asymmetry between individuals and organizations PAAs are designed to reveal and interpret the Blind Self generated by AI inference, transforming privacy management from passive defense to active advocacy

Redefining Privacy as Epistemic Power

The advocacy paradigm redefines privacy as a matter of epistemic power rather than access control This shift calls for a move from system transparency to self-transparency, focusing on how an individual is modeled rather than just how the system operates The framework shifts focus from single disclosures to “inference trajectories” over time, asking users “Do you want to be modeled as Y based on X?” instead of “Do you agree to share X? p. 9

Where the paper says it

PAAs can help users understand how their data might be interpreted and used over time, enabling them to make more informed decisions. Instead of asking users, “Do you agree to share X?

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Personal Advocacy Agents (PAAs) can reduce epistemic asymmetry by revealing and interpreting the Blind Self generated by AI inference, making the inference process more transparent, understandable and even helpful to users.

  2. PAAs can perform an inference audit by leverag[ing] on-device AI to analyze users’ digital footprints in real time, such as browsing history, social media activity, and purchase records, simulating the inferences that external systems might make.

  3. Users can strategically engage with AI outputs by receiving advice such as advising them to selectively disclose this information in specific contexts (such as job screening) to enhance their personal brand or advising them to take measures to suppress the flow of this information, such as data obfuscation or selective deletion of certain data points.

  4. PAAs can help users manage inference trajectories by asking questions like, Do you want to be modeled as Y based on X? instead of just asking, “Do you agree to share X?”, which mitigates deficiencies in existing consent mechanisms.

  5. The system can approximate contextual reasoning by simulating how specific information disclosures propagate through different contexts, providing users with a computational framework for assessing normative violations that would otherwise be difficult to detect.

Abstract

Most privacy regulations function as a passive defensive shield that users must wield themselves. Users are incessantly asked to "opt-in" or "opt-out" of data collection, forced to make defensive decisions whose consequences are increasingly difficult to predict. Viewed through the Johari Window, a psychological framework of self-awareness based on what is known and unknown to self and others, current policies require users to manage information in the Arena and shield the Façade through notice and consent. However, as organizations increasingly use AI to make inferences, the rapid expansion of Blind Spot attributes known to algorithms but unknown to the user emerges as a critical challenge. We illustrate how current regulations fall short and leave this Blind Spot unguarded because they focus on data collection rather than inference. Considering this challenge, we propose a paradigm shift from defensive privacy management to proactive privacy advocacy. We argue for the necessity of Privacy Advocacy Agents capable of simulating possible inferences under social norms to harness the power of AI inference. By illuminating the hidden inferences that users can strategically leverage or suppress, these agents not only restrain the growth of the Blind Spot but also transform the unknown information into personal assets for users. This way, we can foster a flow of personal information that is equitable, transparent, and individually beneficial in the age of AI.

Related papers