MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
MGSM-Pro extends a math reasoning dataset by creating five variations per question across nine languages to test model robustness. Findings show low-resource languages perform poorly when averaged, and model robustness varies significantly by language and model type. The study recommends using the 'Avg-5' setting for reliable mathematical evaluation.