BnMMLU: Measuring Massive Multitask Language Understanding in Bengali
BnMMLU created a massive benchmark of 134,375 Bengali multiple-choice questions across 41 domains to test how well language models understand Bengali. The study found that proprietary models perform best, and enabling reasoning significantly improves performance on hard questions. It highlights the need for high-quality data and specific prompting strategies for better results.