Richness Is Not Evenness: Ordered Renyi Crossings under Self-Training
cs.CL
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/pranjal-srivas/CollisionEntropySelfTraining
Terminology
Sources
- Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
- Synthetic Consumer Insight Generation with Large Language Models
- Where does output diversity collapse in post-training?
- Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies
- Break Step: Recursive Training Resonates with Replayed Sampling Noise
- The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling
- Qwen2.5 Technical Report
- Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs
- What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models
- Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence
- Silent Collapse in Recursive Learning Systems
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data