XDecideBench: Evaluating Typed Decision Models and Confidence Policies Across Languages
cs.CL
Submitted: 2026-11-01
Updated: 2026-11-08
Code: https://github.com/NaiveNeuron/XDecideBench
Project page: https://commoncrawl.github.io/10
Terminology
Sources
- Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
- Evaluating and Benchmarking the System One Model Jev
- SanSi: A Looped Typed Decision Model for System 1.5 Thinking
- Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
- MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages
- Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses
- Benchmarking candidate coverage and rejection policy transfer in typed decision models
- mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
- Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
- Measuring Calibration in Deep Learning
- Benchmarking System One decision models against trained classifiers and language models for automated decision gates
- INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
- MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
- On the Calibration of Multilingual Question Answering LLMs
- Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data