Fine-Tuned Language Models Exhibit Improved Information Conveyance
cs.CL, cs.AI, stat.ML
Submitted: 2026-05-29
Updated: 2026-11-06
Code: https://github.com/WeiyiTian/canopy-entropy
Terminology
Sources
- The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- The Vendi Score: A Diversity Evaluation Metric for Machine Learning
- The Llama 3 Herd of Models
- Measuring Massive Multitask Language Understanding
- Amortizing intractable inference in large language models
- Code Llama: Open Foundation Models for Code
- Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores
- A Long Way to Go: Investigating Length Correlations in RLHF
- On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models
- Optimizing Diversity and Quality through Base-Aligned Model Collaboration
- Finetuned Language Models Are Zero-Shot Learners
- Base Models Beat Aligned Models at Randomness and Creativity
- Qwen3 Technical Report
- LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
- BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data