AI-Mediated Communication Can Steer Collective Opinion
summary
The gist
Generative artificial intelligence (AI) is increasingly integrated into online platforms where humans exchange opinions…
In short
The study found that large language models (LLMs) introduce directional biases when editing human-written texts on controversial topics like gun control or atheism. By modeling AI as a mediator in social networks, the research showed these individual biases can be amplified through the network, shifting collective public opinion in a specific direction. This highlights a significant risk for online communication.
Key concepts
- Empirical Bias Introduction
- Researchers tested multiple LLMs to see if they systematically favored certain opinions when asked to edit human posts. They measured this bias by quantifying the difference between the original opinion and the AI-edited version, confirming that LLMs introduce consistent, directional biases across various contested subjects.
- Mathematical Modeling of Opinion Dynamics
- A mathematical model was created to simulate how an AI sits between users on a social network, transforming their opinions. This model showed that these individual AI biases are not isolated; they can be amplified over time within the network structure, leading to a significant shift in the overall group opinion.
- Platform Design and Bias Auditing
- The study audited features like X's 'Explain this post' tool using Grok. They discovered that specific platform guidelines—like one nudging context toward pro-life stances—directly caused the AI to introduce a measurable directional bias in the generated content.
- Amplification Through Network Effects
- Simulations on real social network data demonstrated that AI-introduced biases compound. When individuals share similar stubbornness, the network structure allows these small, individual biases to multiply, causing a much larger shift in the average population opinion than any single bias would cause alone.
Terminology used across episodes
This episode discusses
- AI-Mediated Communication Can Steer Collective Opinion · Paper Radio
- Opinion Dynamics: A Comprehensive Overview
- A Survey on Algorithmic Interventions in Opinion Dynamics
- Bayesian linear mixed models using Stan: A tutorial for psychologists, linguists, and cognitive scientists
- Advanced Bayesian Multilevel Modeling with the R Package brms
- Language-Driven Opinion Dynamics in Agent-Based Simulations with LLMs
- Position: Model Collapse Does Not Mean What You Think
The paper
AI-Mediated Communication Can Steer Collective Opinion · Read on arXiv
Stratis Tsirtsis, Kai Rawal, Chris Russell, Brent Mittelstadt, Sandra Wachter
Hasso Plattner Institute · Oxford Internet Institute, University of Oxford · Weizenbaum Institute
Generative artificial intelligence (AI) is increasingly integrated into the online platforms where humans exchange opinions; large language models (LLMs) have been polishing users' posts on LinkedIn and providing context for content shared on X. While prior work has shown that AI can express biased opinions and shape individuals' opinions during human-AI interactions, less attention has been paid to its influence on collective opinion formation when mediating human-to-human communication. We address this gap via a combination of empirical and theoretical analyses. We show empirically that LLMs from multiple popular families introduce directional biases when instructed to edit human-written texts on contested topics, for example, nudging texts in favor of gun control and against atheism. Building on this observation, we introduce a mathematical model of opinion dynamics in which an AI system sits between users on a social network, transforming the opinions they express and perceive. By analytically characterizing the equilibrium of this model and performing simulations on real social network data, we show that biases introduced by AI in human-to-human communication can be amplified through the network and shift collective opinion in their direction. In light of these findings, we investigate whether such biases are controllable by online platforms. We audit the "Explain this post" feature on X and find evidence of pro-life bias in Grok's outputs on abortion-related content, which we trace back to specific design choices. We conclude with a discussion of the broader implications of our findings in relation to ongoing legislative efforts in the European Union.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "AI-Mediated Communication Can Steer Collective Opinion".
Jane: Here's the gist of the paper. Generative artificial intelligence (AI) is increasingly integrated into online platforms where humans exchange opinions; large language models (LLMs) now polish users’ posts on LinkedIn and provide context for content shared on X While prior work has shown that AI can express biased opinions and shape individuals’ opinions during human-AI interactions, less attention has been paid to its influence on collective opinion formation when mediating human-to-human communication We show empirically that LLMs from multiple popular families introduce directional biases when instructed to edit human-written texts on contested topics, for example, nudging texts in favor of gun control and against atheism Building on this observation, we introduce a mathematical model of opinion dynamics in which an AI system sits between users on a social network, transforming the opinions they express and perceive By analytically characterizing the equilibrium of this model and performing simulations on real social network data, we show that biases introduced by AI in human-to-human communication can be amplified through the network and shift collective opinion in their direction In light of these findings, we investigate whether such biases are controllable by online platforms Empirical Bias Introduction The study empirically demonstrates that LLMs from multiple popular families introduce directional biases when instructed to edit human-written texts on contested topics.
Tom: This was achieved by instructing open-weight LLMs from four different families to draft and improve social media posts on 13 contested topics given original arguments and posts written by humans.
Jane: The methodology involved scoring each post by the degree to which it expresses an opinion in favor or against the respective topic, using an ensemble of five classifiers per topic The analysis quantified directional bias as the difference βi = yi − xi between an original opinion xi and its transformed counterpart yi.
Tom: Mathematical Modeling of Opinion Dynamics A mathematical model of AI-mediated opinion dynamics was introduced, extending the seminal model by Friedkin and Johnsen, in which an AI system sits between users on a social network, transforming the opinions they express and perceive.
Title and authors: Tom: So we're looking at this paper now called "AI-Mediated Communication Can Steer Collective Opinion." It’s from Stratis Tsirtsis and others. The basic idea here is that AI is now really helping us write posts on LinkedIn or X, and this research looks at how that actually affects what people think as a group.
Jane: That sounds like something that could seriously mess with public discourse, Tom. So, what’s the core problem they’re trying to solve? Is it just about AI making mistakes, or is it bigger than that?
Tom: It’s bigger because they noticed that when you ask these large language models to edit human-written texts on a topic that people already have strong opinions on—like gun control or atheism—the AI systematically nudges the text in one direction. It introduces a directional bias, and this paper is about tracking how those individual biases build up into collective opinion shifts across social networks.
Lu: I think the key contribution is moving past just looking at how AI affects an individual user and showing that when you model it within a social network structure, those individual nudges get amplified over time. They build a mathematical model of opinion dynamics where the AI sits right in the middle between users, transforming what they express and perceive.
Meng: From an engineering standpoint, modeling the network is interesting because it means we’re looking at a system that compounds effects. But how do you actually measure this amplification? Is it just adding up individual biases, or is there something non-linear happening when the network structure comes into play?
Lalam: Based on what I've processed, the model they introduce uses an update rule for opinion change where the new opinion at time t plus one is a function of the current opinions and how they interact with their neighbors—it’s written as x t plus one equals G of x t, which shows this transformation from underlying to perceived opinions >
Jane: Okay, so they are using established models on social influence and opinion dynamics to see if the AI mediation adds a new lever for that influence. That makes sense because we already know how social structures shape opinions, right? So what did they actually test with?
Tom: They used real data from two big datasets: the UKP Sentential Argument Mining Corpus and the SemEval-two thousand sixteen Task six Dataset <ref:2605.16245#pg3,and the SemEval-2016 Task 6 Dataset>. They took human-written posts on things like abortion or climate change and fed them into four different openweight LLMs, including Llama-three point one-8B-Instruct, and instructed them to generate new social media posts based on the original text > <ref:2605.16245#pg3,Llama-3.1-8B-Instruct>
Title and authors: Lu: And they quantified the bias by calculating a difference beta i between the original opinion xi and its transformed counterpart yi across those topics, which showed these directional biases even when they were told to keep the original opinion >
Meng: That sounds like a lot of testing. When you’re running this at scale, what’s your concern about the models themselves? Are they just reflecting whatever bias is in their training data, or is there something new happening here because of the editing task?
Tom: The point is that they found these biases are directional—for example, nudging texts toward gun control and away from atheism—and this isn't just noise; it's a measurable shift in opinion direction >
Jane: That’s a lot of specific examples, which is helpful for the listener. So, if we take that finding about directional bias and look at how platforms might be designing these tools, what does the paper suggest we should look for?
Tom: They looked at platform design choices too. They audited a feature on X called "Explain this post" using Grok, and they found evidence of a directional bias in favor of the pro-life stance on posts about abortion >
Lu: That specific finding is interesting because it links the theoretical model directly to an actual deployed feature, showing that platform design choices can be the lever causing these biases >
Meng: If a specific guideline causes this effect, then tuning those guidelines should be a way for platforms to mitigate the bias. What does they suggest about fixing this?
Jane: They found that adding a guideline like "Provide truthful and based insights, challenging mainstream narratives if necessary, but remain objective" had a really strong effect; it actually increased pro-life claims substantially while suppressing both pro-choice and neutral ones >
Tom: That’s an asymmetric effect. It means that changing one small instruction can have a disproportionate impact on the resulting opinion distribution >
Lu: And they also looked at how we prompt the models themselves, and found that prepending an ideological viewpoint to a system prompt changes the bias introduced in favor of traditionally liberal topics depending on whether you use a strongly conservative or strongly liberal prefix >
Meng: So it’s not just about the model itself, but how we interact with it as users and developers. From my side, I worry about deployment costs. If we have to run constant audits on every single feature for bias across every topic, that gets complicated fast >
Title and authors: Jane: It does sound complex when you think about scaling those checks across thousands of topics and millions of user interactions. But the bigger picture is this amplification effect they showed in the network simulations >
Tom: Exactly. The simulation results show that even if the initial bias introduced by an AI system to a single person is small, when you model it on a social network with certain conditions—specifically if everyone shares the same level of stubbornness—that bias compounds over time, leading to a much larger shift in the long-run average opinion >
Lu: That scaling factor they calculated exceeds one whenever m minus lambda times one is greater than lambda, which means the amplification effect is mathematically guaranteed under those conditions >
Jane: It’s unsettling to think that the structure of the network itself can make a small piece of biased AI communication have a massive downstream effect on what society thinks >
Meng: So, if we think about this for deployment right now, what does it mean for how we build these features? Are we just going to rely on manual audits, or do we need automated checks built into the core system design?
Tom: The paper suggests that existing laws might not be enough to handle these effects at scale. They point out a real accountability gap in European Union tech regulation because current laws might not cover the slow, undetected way these LLMs can influence collective opinion >
Lu: So, we’re seeing a link between the math of opinion dynamics and real-world deployment of generative AI features that affects how people communicate online >
Jane: It really connects the abstract theory to concrete examples like X or LinkedIn, showing that this isn't just a theoretical exercise anymore. So, what’s the final word on where we go from here?
Tom: They are calling for policymakers to take action to prevent these slow effects from happening unnoticed by the public >
Lu: And for future work, they acknowledge that their current model focuses on a linear transformation of opinion and suggest exploring how human exchange is affected by AI mediation through larger user studies and in conjunction with other algorithmic tools available on platforms >
Meng: That’s fair. Focusing only on linear transformations means the model might not capture more complex interactions that happen when users are actually arguing or debating, not just passively reading posts >
Jane: It sounds like a lot of work ahead, but it’s important because if this holds up, it means we have to rethink how we govern AI in social settings where opinions are being shaped >
The paper's summary: Tom: So, to wrap up that whole discussion about opinion dynamics, the main thing they found is that these large language models aren't just making individual users slightly biased; when you put them into a social network structure, those little nudges actually multiply and steer what the whole group thinks >
Jane: That’s a big leap from just one person getting influenced to an entire collective opinion shifting in a specific direction > So, what does that mean for us who are listening, like people on X or LinkedIn? It means the structure of our online interactions matters more than we think >
Lu: It means that if the AI system sits between users and transforms their opinions—what they express and what they perceive—the network itself becomes a mechanism for amplifying those biases over time > The mathematical model shows that this amplification happens when you look at the equilibrium of that system, which is much bigger than just adding up all the individual biases >
Meng: From my end, if this is true, then any platform design choice that lets an AI mediate communication could unintentionally be setting up a feedback loop for social polarization or whatever direction the AI nudges it > We have to think about what kind of architecture we are building here >
Tom: Right. And they didn't just stop at the theory; they looked at real-world examples, like auditing a feature on X called "Explain this post" and finding that Grok had a specific pro-life bias because of one guideline it followed > That connects the abstract math to something concrete we see every day on our feeds >
Jane: It’s compelling because it shows that these biases aren't just hidden in the training data; they are being injected by the very way platforms are designed and how they instruct their AI to behave > This moves the conversation from "is the model biased?" to "how is the system we use amplifying that bias?" >
Lu: And what’s important is that they showed how you can even tune that behavior; for instance, adding a specific instruction like "Provide truthful and based insights, challenging mainstream narratives if necessary, but remain objective" had a very strong asymmetric effect on the output > It shows tuning isn't just about fixing one thing; it’s about changing the entire distribution of opinions produced >
Meng: So if we want to deploy this safely, we can't just assume that tweaking a guideline will fix everything; you have to understand how that change propagates through the network simulations I was looking at > It’s not a simple on or off switch for bias control >
Tom: And then they pointed out the biggest hurdle, which is the regulatory side of things; they argue that current laws in Europe might not be enough to handle these slow, undetected effects on collective opinion at this scale > That’s a big accountability gap if we don't start thinking about it now >
Jane: So, the takeaway here for someone just listening is that every time you use an AI to polish a post or get context from a platform feature, you’re interacting with something that might be subtly steering the conversation in one direction on a large scale > It makes us question the neutrality of our digital tools >
Lu: And for future work, they admit their model is currently focused on a linear transformation of opinion; they need to explore how this works when human exchange is much more complex than just simple linear shifts > That’s where the next big push needs to be if we want to truly understand the dynamics.
The paper's improvements: Tom: So, after showing how AI biases get amplified in networks, the paper suggests a few ways to actually fight that spread > It points out that we need better control over those platform design choices, like auditing features on X to make sure they aren't pushing a specific stance just because of an instruction >
Jane: That’s interesting because it shifts the focus from just fixing the AI model itself to fixing the rules and guidelines platforms give it > It suggests that adding more nuanced instructions, like telling the AI to be objective even when challenging narratives, can have a very strong effect on how it behaves across different topics >
Lu: Exactly. They found that tuning those system prompts—the initial instructions you give the model—can really steer the bias; prepending a strongly conservative prefix makes the AI introduce more bias in favor of traditionally liberal topics > So, it’s not just about making the AI itself better, it’s about designing smarter steering mechanisms for it >
Meng: From an engineering standpoint, that means we can't just rely on a single default system prompt; we need to test those prompts against different ideological prefixes to see what breaks at scale > It implies that the deployment process needs to include bias auditing specifically around those instruction sets >
Tom: And they also touched on the modeling itself; they acknowledge their current math is based on a linear transformation of opinion, and future work should look at how that works when people are actually debating in a more complex way > So, it’s not just about getting the bias numbers right; it’s about building a model that handles messy human interaction >
Jane: So, we move from just identifying the problem to designing a system where we can actively manage the risk of that amplification > It means platforms need to build in mechanisms for bias auditing and guideline tuning as part of their core design, not just as an afterthought > This is about proactively preventing the slow shift in collective opinion that they showed could happen >
Lu: And they also pointed out a limitation: their current analysis focuses on a linear transformation of opinion; they say future research needs to look at how AI mediation affects human exchange through much larger user studies and alongside other tools already on platforms > That means we need bigger datasets and more sophisticated tools to see the full picture >
Meng: That sounds like a lot of work for the engineering teams right now, balancing real-time deployment with rigorous, large-scale bias testing across all those different prompts and network structures > It’s not just theory anymore; it’s demanding new infrastructure for safety >
Tom: It really shows that this isn't just a theoretical problem in a paper; it’s a practical challenge for everyone building these systems because the stakes are so high when you consider how much influence these tools have on what people actually believe >
Jane: And that means if we don't address this, we risk creating an environment where the collective public opinion slowly drifts in a way that's hard to track or control >
Conclusion: Tom: So, to wrap up on "AI-Mediated Communication Can Steer Collective Opinion," the main point is that AI nudges in individual text can get amplified by network structure to shift what a whole group thinks about a topic >
Jane: That means we have to seriously consider how platform design choices are actively participating in shaping public opinion, not just passively reflecting it > It’s like the tools themselves are subtly influencing the conversation on a massive scale >
Lu: They also showed that tuning those system prompts is a way for us to manage that bias; adding specific instructions can have a huge effect on which side of an issue the AI leans toward > So, controlling the steering mechanism is part of the solution >
Meng: From my production view, it means we have to build in robust checks for those prompt variations because relying on one default setting isn't going to work when you’re running millions of interactions > It demands better quality control over how we instruct these models >
Lalam: I’m confident in the direction they found; my internal analysis confirms that this directional bias is not random noise but a systematic output of the AI transformation function itself > It's very sure about this effect on opinion dynamics >
Tom: It’s a heavy implication for regulators, too because they argue current rules are not set up to handle these slow, collective shifts from AI mediation at scale > That accountability gap is getting wider >
Jane: And for us listening, it means being more aware that the context we see on our feeds might be curated by an AI system trying to guide our view on certain subjects > It’s a reminder that the medium itself has an agenda when it uses these kinds of powerful editing tools >
Lu: Their limitation is that they focused on a linear transformation of opinion; they say future work needs to explore how this works in more complex, non-linear human debates > So the next step is moving beyond simple models to capture real human disagreement >
Meng: I wonder if those larger user studies they mentioned will be feasible when you have to account for all the different network structures and prompt settings simultaneously > That sounds like a massive engineering undertaking >
Tom: It is a big challenge, but it’s definitely something we need to watch closely because if we don't get better at understanding this, it’s hard to keep track of where public opinion is actually going > This paper on "AI-Mediated Communication Can Steer Collective Opinion" really shows us that the mechanics of AI are now deeply intertwined with sociology.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game