Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias
summary
The gist
Generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten…
In short
This study compares Wikipedia and Grokipedia, a generative encyclopedia, focusing on how AI systems handle encyclopedic content. The research found that Grokipedia largely maintains the structural organization of reference knowledge but actively intervenes in how content is selected, rewritten, and framed.
Key concepts
- Content Selection
- This refers to the process where the system decides which existing reference pages to include in its output. The researchers suggest improving this by using a model that looks beyond simple popularity and considers other factors like how often a page is referenced or how much editorial work has been done on it.
- Textual Rewriting
- This involves modifying the actual text of an encyclopedia entry. The study indicates that rewriting should be prioritized for topics that have a high density of references and are currently subjects of recent controversy, as these areas benefit most from revision.
- Evaluative Framing Moderation
- This is the system's ability to control the tone and language used in an entry. The goal is to reduce conflict-oriented language by using an LLM prompt designed to minimize praise or glorification, steering the framing toward a neutral middle ground.
- Narrative Structure Preservation
- This concept addresses keeping the overall organization of information consistent. The researchers want to ensure that the relationships between different entities (actor-relation networks) remain similar between what Grokipedia generates and the original reference sources.
Terminology used across episodes
This episode discusses
- Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias · Paper Radio
- Why Language Models Hallucinate
- Epistemological Fault Lines Between Human and Artificial Intelligence
- How Similar Are Grokipedia and Wikipedia? A Multi-Dimensional Textual and Structural Comparison
- What did Elon change? A comprehensive analysis of Grokipedia
- Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority · Paper Radio
- Is Grokipedia Right-Leaning? Comparing Political Framing in Wikipedia and Grokipedia on Controversial Topics
The paper
Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias · Read on arXiv
Ortal Hadad, Edoardo Loru, Jacopo Nudo, Anita Bonetti, Matteo Cinelli, Walter Quattrociocchi
Department of Computer Science, Sapienza University of Rome · Department of Computer, Control and Management Engineering, Sapienza University of Rome · Department of Communication and Social Research, Sapienza University of Rome
We present a comparative analysis of Wikipedia and Grokipedia to examine how generative mediation alters content selection, textual rewriting, narrative structure, and evaluative framing in encyclopedic content. We model page inclusion in Grokipedia as a function of Wikipedia page popularity, density of reference, and recent editorial activity. Inclusion is non-uniform: pages with higher visibility and greater editorial conflict in Wikipedia are more likely to appear in Grokipedia. For included pages, we distinguish between verbatim reproduction and generative rewriting. Rewriting is more frequent for pages with higher reference density and recent controversy, while highly popular pages are more often reproduced without modification. We compare editing activity across the two platforms and estimate page complexity using a fitness-complexity framework to assess whether generative mediation alters patterns of editorial participation. To assess narrative organization, we construct actor-relation networks from article texts using abstract meaning representation. Across multiple topical domains, including U.S. politics, geopolitics, and conspiracy-related narratives, narrative structure remains largely consistent between the two sources. Analysis of lead sections shows broadly correlated framing, with localized shifts in laudatory and conflict-oriented language for some topics in Grokipedia. Overall, generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten, and framed.
DOI: 10.1145/3840301
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Wikipedia and Grokipedia".
Jane: Here's the gist of the paper. Generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten, and framed The analysis reveals that Grokipedia largely preserves the structural organization of reference knowledge while intervening in content selection, rewriting, and consolidation Content Selection and Rewriting Signals
Title and authors: Tom: Let's go over the summary section of "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" to see their main findings on selection, rewriting, structure, and framing.
Jane: Basically, they set up a direct comparison between Wikipedia’s existing revision process and how Grokipedia uses AI to mediate that content.
Lu: The summary points out that empirical evidence from controlled tasks and simulations shows that generative agents can amplify features already in the source material while introducing structural distortions compared to how humans actually behave.
Meng: So even when the overall structure looks similar, the way something is framed or rewritten could be subtly different depending on whether you look at Wikipedia or Grokipedia.
Lalam: They found that generative systems tend to preserve the main structural organization of encyclopedic content across all the topics they tested.
Tom: That's a significant finding—that even though the way pages are chosen and written is different, the overall shape of the reference knowledge stays consistent.
Jane: It seems their focus isn't really on whether a specific fact is true, but more on the process of organizing knowledge when an AI steps in to assemble and present existing reference material.
Lu: The paper tackles that gap by looking at Grokipedia as this generative mediation layer built on top of Wikipedia, focusing on how it impacts content selection and rewriting rather than just checking factual correctness.
Meng: It’s about studying the generative system at the level of encyclopedic content assembly, which is a very specific way to approach this kind of problem.
Lalam: They are analyzing precisely how content gets selected, rewritten, and stabilized when these systems mediate information that originally came from human curators.
The paper's summary: Tom: Now let's look at the suggestions in "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" for how to make the Grokipedia system better.
Jane: They suggest improving content selection by moving away from just relying on page popularity and instead using a multivariate model that accounts for correlations between page views, reference density, and editorial activity.
Lu: That would mean selecting pages more uniformly instead of letting just raw popularity dictate inclusion, which is where they felt the system was too biased toward what’s already popular.
Meng: From my point of view, if you’re going to select content for production, you need a model that actually looks at density and activity alongside just raw view counts.
Lalam: They also suggest refining textual rewriting by prioritizing pages with higher reference density and recent controversy because those are where they observed rewriting happening most frequently.
Tom: So it’s not just about picking popular stuff; it’s about targeting pages that are dense and actively being discussed when rewriting is needed.
Jane: And there's also a suggestion to moderate evaluative framing by tuning the system to reduce conflict-oriented language in the lead section, aiming for a score closer to the quadrant bisector shown in their figure.
Lu: That’s about making sure the language isn't overly biased or praising certain viewpoints, which connects back to how we study evaluative framing.
Meng: I wonder if that moderation is actually feasible when you’re trying to keep accuracy up while filtering out conflict language; it sounds like a real balancing act.
Lalam: All these improvements aim to keep the structural organization sound while making the process of content generation more nuanced and less likely to introduce certain kinds of distortion.
The paper's improvements: Tom: So wrapping up on "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias," the paper concludes that generative systems preserve the main structural organization while still intervening in selection, rewriting, and framing.
Jane: They summarize that even with all these interventions, there’s still a lot we need to figure out about how this mediation actually affects the actual knowledge base.
Lu: The implication is that studying content selection and stabilization when generative systems mediate information from human sources is a critical area for research right now.
Meng: It suggests that for any real-world deployment of an AI encyclopedia, you need mechanisms to manage how content gets picked and presented to avoid unintended distortions as the system scales up.
Lalam: And they point out a limitation: the work focuses on analyzing these processes rather than providing a fully deployed, universally accurate system.
Tom: Exactly, so it’s a strong analysis of how the mechanism works but not necessarily a final verdict on factual perfection across every single domain.
Jane: It’s a big step in understanding how AI systems interact with established knowledge structures, showing that the process matters as much as the content itself.
Lu: This sets up future work where we can look at those specific selection and rewriting improvements they suggested to see if those changes actually yield better outcomes.
Meng: I think for production, we need to see how robust these selection models are when faced with truly messy, real-world data that doesn't perfectly mirror the controlled setup of the study.
Lalam: So "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" gives us a clear map of where generative mediation is happening in encyclopedic content today.
Conclusion: Tom: So we've looked at "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias," and the big takeaway is that generative systems keep the main structure of reference knowledge intact, but they definitely change how content is chosen, rewritten, and framed.
Jane: It really shows us that even when an AI is assembling a Wikipedia-like page, we still need to pay attention to how it picks things out of the crowd and how it shapes the narrative around those facts.
Lu: The implication here is that studying how content gets selected and stabilized when generative systems mediate information from human sources is a critical area for research right now.
Meng: For anyone building a real-world AI encyclopedia, this suggests you need mechanisms to manage what content gets picked and presented to avoid big distortions at scale.
Lalam: They also pointed out that the work focuses on analyzing these processes rather than giving us a finished, perfectly accurate system for everything.
Tom: Exactly, so it’s a strong analysis of the mechanism but not necessarily a final verdict on factual perfection across all domains.
Jane: It's a big step in understanding how AI systems interact with established knowledge structures because it shows that the process matters as much as the content itself.
Lu: This sets up future work where we can look at those specific selection and rewriting improvements they suggested to see if those changes actually yield better outcomes.
Meng: I think for production, you need to see how robust these selection models are when faced with really messy, real-world data that doesn't perfectly mirror the controlled setup of the study.
Lalam: So "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" gives us a clear map of where generative mediation is happening in encyclopedic content today.
Tom: It’s definitely a fascinating look at how we build knowledge systems, but next up, we're moving on to some completely different stuff with polynomial-time computation for fixed points.
More episodes
- 2411.16152-Optimizing Winograd Convolution on ARMv8 processors
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines