Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Wikipedia and Grokipedia".
Jane: Here's the gist of the paper. Generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten, and framed The analysis reveals that Grokipedia largely preserves the structural organization of reference knowledge while intervening in content selection, rewriting, and consolidation Content Selection and Rewriting Signals
Title and authors: Tom: Let's go over the summary section of "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" to see their main findings on selection, rewriting, structure, and framing.
Jane: Basically, they set up a direct comparison between Wikipedia’s existing revision process and how Grokipedia uses AI to mediate that content.
Lu: The summary points out that empirical evidence from controlled tasks and simulations shows that generative agents can amplify features already in the source material while introducing structural distortions compared to how humans actually behave.
Meng: So even when the overall structure looks similar, the way something is framed or rewritten could be subtly different depending on whether you look at Wikipedia or Grokipedia.
Lalam: They found that generative systems tend to preserve the main structural organization of encyclopedic content across all the topics they tested.
Tom: That's a significant finding—that even though the way pages are chosen and written is different, the overall shape of the reference knowledge stays consistent.
Jane: It seems their focus isn't really on whether a specific fact is true, but more on the process of organizing knowledge when an AI steps in to assemble and present existing reference material.
Lu: The paper tackles that gap by looking at Grokipedia as this generative mediation layer built on top of Wikipedia, focusing on how it impacts content selection and rewriting rather than just checking factual correctness.
Meng: It’s about studying the generative system at the level of encyclopedic content assembly, which is a very specific way to approach this kind of problem.
Lalam: They are analyzing precisely how content gets selected, rewritten, and stabilized when these systems mediate information that originally came from human curators.
The paper's summary: Tom: Now let's look at the suggestions in "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" for how to make the Grokipedia system better.
Jane: They suggest improving content selection by moving away from just relying on page popularity and instead using a multivariate model that accounts for correlations between page views, reference density, and editorial activity.
Lu: That would mean selecting pages more uniformly instead of letting just raw popularity dictate inclusion, which is where they felt the system was too biased toward what’s already popular.
Meng: From my point of view, if you’re going to select content for production, you need a model that actually looks at density and activity alongside just raw view counts.
Lalam: They also suggest refining textual rewriting by prioritizing pages with higher reference density and recent controversy because those are where they observed rewriting happening most frequently.
Tom: So it’s not just about picking popular stuff; it’s about targeting pages that are dense and actively being discussed when rewriting is needed.
Jane: And there's also a suggestion to moderate evaluative framing by tuning the system to reduce conflict-oriented language in the lead section, aiming for a score closer to the quadrant bisector shown in their figure.
Lu: That’s about making sure the language isn't overly biased or praising certain viewpoints, which connects back to how we study evaluative framing.
Meng: I wonder if that moderation is actually feasible when you’re trying to keep accuracy up while filtering out conflict language; it sounds like a real balancing act.
Lalam: All these improvements aim to keep the structural organization sound while making the process of content generation more nuanced and less likely to introduce certain kinds of distortion.
The paper's improvements: Tom: So wrapping up on "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias," the paper concludes that generative systems preserve the main structural organization while still intervening in selection, rewriting, and framing.
Jane: They summarize that even with all these interventions, there’s still a lot we need to figure out about how this mediation actually affects the actual knowledge base.
Lu: The implication is that studying content selection and stabilization when generative systems mediate information from human sources is a critical area for research right now.
Meng: It suggests that for any real-world deployment of an AI encyclopedia, you need mechanisms to manage how content gets picked and presented to avoid unintended distortions as the system scales up.
Lalam: And they point out a limitation: the work focuses on analyzing these processes rather than providing a fully deployed, universally accurate system.
Tom: Exactly, so it’s a strong analysis of how the mechanism works but not necessarily a final verdict on factual perfection across every single domain.
Jane: It’s a big step in understanding how AI systems interact with established knowledge structures, showing that the process matters as much as the content itself.
Lu: This sets up future work where we can look at those specific selection and rewriting improvements they suggested to see if those changes actually yield better outcomes.
Meng: I think for production, we need to see how robust these selection models are when faced with truly messy, real-world data that doesn't perfectly mirror the controlled setup of the study.
Lalam: So "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" gives us a clear map of where generative mediation is happening in encyclopedic content today.
Conclusion: Tom: So we've looked at "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias," and the big takeaway is that generative systems keep the main structure of reference knowledge intact, but they definitely change how content is chosen, rewritten, and framed.
Jane: It really shows us that even when an AI is assembling a Wikipedia-like page, we still need to pay attention to how it picks things out of the crowd and how it shapes the narrative around those facts.
Lu: The implication here is that studying how content gets selected and stabilized when generative systems mediate information from human sources is a critical area for research right now.
Meng: For anyone building a real-world AI encyclopedia, this suggests you need mechanisms to manage what content gets picked and presented to avoid big distortions at scale.
Lalam: They also pointed out that the work focuses on analyzing these processes rather than giving us a finished, perfectly accurate system for everything.
Tom: Exactly, so it’s a strong analysis of the mechanism but not necessarily a final verdict on factual perfection across all domains.
Jane: It's a big step in understanding how AI systems interact with established knowledge structures because it shows that the process matters as much as the content itself.
Lu: This sets up future work where we can look at those specific selection and rewriting improvements they suggested to see if those changes actually yield better outcomes.
Meng: I think for production, you need to see how robust these selection models are when faced with really messy, real-world data that doesn't perfectly mirror the controlled setup of the study.
Lalam: So "Wikipedia and Grokipedia: A Comparison of Human and Generative Encyclopedias" gives us a clear map of where generative mediation is happening in encyclopedic content today.
Tom: It’s definitely a fascinating look at how we build knowledge systems, but next up, we're moving on to some completely different stuff with polynomial-time computation for fixed points.
Ortal Hadad, Edoardo Loru, Jacopo Nudo, Anita Bonetti, Matteo Cinelli, Walter Quattrociocchi
Department of Computer Science, Sapienza University of Rome · Department of Computer, Control and Management Engineering, Sapienza University of Rome · Department of Communication and Social Research, Sapienza University of Rome
cs.CY, cs.SI, physics.soc-ph
Submitted: 2026-02-05
Updated: 2026-02-05
Journal ref: ACM Trans. Web 21, 1, Article 14 (February 2027)
DOI: 10.1145/3840301
Code: https://github.com/EFC-data/fermi
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 80/100
The gist: Generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten…
Key concepts
- Content Selection
- This refers to the process where the system decides which existing reference pages to include in its output. The researchers suggest improving this by using a model that looks beyond simple popularity and considers other factors like how often a page is referenced or how much editorial work has been done on it.
- Textual Rewriting
- This involves modifying the actual text of an encyclopedia entry. The study indicates that rewriting should be prioritized for topics that have a high density of references and are currently subjects of recent controversy, as these areas benefit most from revision.
- Evaluative Framing Moderation
- This is the system's ability to control the tone and language used in an entry. The goal is to reduce conflict-oriented language by using an LLM prompt designed to minimize praise or glorification, steering the framing toward a neutral middle ground.
- Narrative Structure Preservation
- This concept addresses keeping the overall organization of information consistent. The researchers want to ensure that the relationships between different entities (actor-relation networks) remain similar between what Grokipedia generates and the original reference sources.
Terminology
Summary
Generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten, and framed The analysis reveals that Grokipedia largely preserves the structural organization of reference knowledge while intervening in content selection, rewriting, and consolidation
Content Selection and Rewriting Signals
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Content selection improvement: The system can be improved to select pages more uniformly rather than based solely on popularity, by applying a multivariate model that accounts for confounding correlations between page views, reference density, and editorial activity when predicting inclusion in Grokipedia.
-
Textual rewriting refinement: Rewriting should be prioritized for pages with higher reference density and recent controversy, as this is where
Rewriting is more frequent for pages with higher reference density and recent controversy.
-
Evaluative framing moderation: The system can be tuned to reduce conflict-oriented language by analyzing the lead section's framing using an LLM prompt designed to minimize praise or glorification, aiming for a score closer to the quadrant bisector in Fig. 4.
-
Narrative structure preservation: The system can maintain the
core narrative organization of existing reference content
by ensuring that actor-relation networks remain largely consistent between the generated and reference sources, as observed across multiple topical domains.
Abstract
We present a comparative analysis of Wikipedia and Grokipedia to examine how generative mediation alters content selection, textual rewriting, narrative structure, and evaluative framing in encyclopedic content. We model page inclusion in Grokipedia as a function of Wikipedia page popularity, density of reference, and recent editorial activity. Inclusion is non-uniform: pages with higher visibility and greater editorial conflict in Wikipedia are more likely to appear in Grokipedia. For included pages, we distinguish between verbatim reproduction and generative rewriting. Rewriting is more frequent for pages with higher reference density and recent controversy, while highly popular pages are more often reproduced without modification. We compare editing activity across the two platforms and estimate page complexity using a fitness-complexity framework to assess whether generative mediation alters patterns of editorial participation. To assess narrative organization, we construct actor-relation networks from article texts using abstract meaning representation. Across multiple topical domains, including U.S. politics, geopolitics, and conspiracy-related narratives, narrative structure remains largely consistent between the two sources. Analysis of lead sections shows broadly correlated framing, with localized shifts in laudatory and conflict-oriented language for some topics in Grokipedia. Overall, generative systems preserve the main structural organization of encyclopedic content, while affecting how content is selected, rewritten, and framed.
Sources
- Why Language Models Hallucinate
- Epistemological Fault Lines Between Human and Artificial Intelligence
- How Similar Are Grokipedia and Wikipedia? A Multi-Dimensional Textual and Structural Comparison
- What did Elon change? A comprehensive analysis of Grokipedia
- Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority
- Is Grokipedia Right-Leaning? Comparing Political Framing in Wikipedia and Grokipedia on Controversial Topics
Related papers
- AI-Mediated Communication Can Steer Collective Opinion
- Epistemic Infrastructures of Science in AI Era Should Rebalance Costs of Generation and Verification
- AI out-persuades expert humans in text-based dialogue
- Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data
- AI Value Alignment for Evolving Social Norms
- Voting Biases in Decentralized Autonomous Organization (DAO) Governance