Beyond Polarization: The Generative Constraint of Chain-of-Thought in Pointwise Reranking
This episode discusses the paper "Beyond Polarization," which examines the stable performance gap between Chain-of-Thought (CoT) models and direct scoring models. Researchers found this gap persists regardless of model size, attributing it to a "discrete text bottleneck" that causes information loss. This suggests a fundamental design limitation requiring architectural solutions over simple training improvements.