LLM Validation Makes Minimal-Pair Benchmarks Easier for Language Models
cs.CL
Submitted: 2026-11-10
Updated: 2026-11-11
Terminology
Sources
- QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
- GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction
- MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
- CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models
- LLM Evaluators Recognize and Favor Their Own Generations
- FreqBLiMP: Frequency-Controlled Minimal Pairs Reveal Robustness and Fragility of LLMs Under Lexical Rarity
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data