Same Rule, Different Verdicts: Rubric Language and Clause Structure in LLM Judges for Korean Support Dialogues
cs.CL
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 16 pages, 2 figures. Data and code are included as ancillary files
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Judging LLM-as-a-Judge: Concerning Rubric Artifacts in LLM-based Automated Text Generation Evaluation
- Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
- Cross-Lingual LLM-Judge Transfer via Evaluation Decomposition
- MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Study
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data