Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs
The episode discusses 'Estimating Uncertainty from Reasoning,' analyzing how LLM performance varies across twenty-two languages and model sizes. Key findings show that prompting models in English boosts performance on low-resource languages, and that uncertainty measurement methods must change—from open-box for small models to closed-box for large ones.