BhashaBench V1: A Comprehensive Benchmark for the Quadrant of Indic Domains
cs.CL, cs.AI
Submitted: 2025-10-29
Updated: 2025-10-30
Code: https://github.com/AI4Bharat/IndicLID
Project page: https://bharatgen-iitb-tih.github.io/bhashabenchv1
Terminology
Sources
- MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
- Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective
- Leveraging Vision Language Models for Specialized Agricultural Tasks
- Lessons from the Trenches on Reproducible Evaluation of Language Models
- Language Models are Few-Shot Learners
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- LEGAL-BERT: The Muppets straight out of Law School
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
- DeepSeek-V3 Technical Report
- IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
- Airavata: Introducing Hindi Instruction-tuned LLM
- The Llama 3 Herd of Models
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- OpenTCM: A GraphRAG-Empowered LLM-based System for Traditional Chinese Medicine Knowledge Retrieval and Diagnosis
- Measuring Massive Multitask Language Understanding
- FinanceBench: A New Benchmark for Financial Question Answering
- Mistral 7B
- IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning
- MuRIL: Multilingual Representations for Indian Languages
- MTCMB: A Multi-Task Benchmark Framework for Evaluating LLMs on Knowledge, Reasoning, and Safety in Traditional Chinese Medicine
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data