A Cognitive Framework for Idea Diversity in Large Language Models
cs.CY, cs.AI, cs.HC
Submitted: 2026-02-23
Updated: 2026-11-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: Consumers, marketers, and new product developers increasingly rely on LLMs for ideation.
Terminology
Abstract
Consumers, marketers, and new product developers increasingly rely on LLMs for ideation. Yet ideas generated by independent LLM sessions tend to be less diverse than ideas generated by independent samples of humans, raising concerns that widespread reliance on LLMs could homogenize marketplace ideation and decrease consumer welfare. We identify two mechanisms undermining LLM idea diversity, informed by theories of creative ideation from new product development and cognitive psychology. First, LLMs aggregate knowledge into a unified distribution rather than exhibiting the knowledge partitioning inherent to human populations, where each person occupies a distinct region of the knowledge space. Second, at the individual level, LLMs exhibit fixation just as humans do, where early outputs constrain subsequent ideation. Two product-ideation experiments demonstrate that targeted prompting interventions can address both mechanisms. Persona modifiers based on ordinary individuals improve knowledge partitioning but amplify fixation. This fixation can be mitigated by using chain-of-thought (CoT) prompting, and combining both strategies produces the highest idea diversity, outperforming humans. These findings offer a theoretically grounded framework for understanding LLM idea diversity and practical strategies for leveraging AI's efficiency without compromising the idea diversity that sustains marketplace variety and consumer welfare.
Sources
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- Prompting Diverse Ideas: Increasing AI Idea Variance
- GPT-4 Technical Report
- AI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion
- Is Temperature the Creativity Parameter of Large Language Models?
- Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
- Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
Related papers
- AI-Mediated Communication Can Steer Collective Opinion
- Epistemic Infrastructures of Science in AI Era Should Rebalance Costs of Generation and Verification
- AI out-persuades expert humans in text-based dialogue
- Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data
- AI Value Alignment for Evolving Social Norms
- Voting Biases in Decentralized Autonomous Organization (DAO) Governance