JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
cs.CL, cs.AI
Submitted: 2026-05-26
Updated: 2026-11-14
Project page: https://jinjh0123.github.io/JuICE
Terminology
Sources
- Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking
- LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output
- gpt-oss-120b & gpt-oss-20b Model Card
- OpenAI GPT-5 System Card
- MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data