HakemBench: A Turkish Benchmark of Typed Decisions
cs.CL
Submitted: 2026-10-01
Updated: 2026-10-01
Code: https://github.com/ufakai/hakembench
Terminology
Sources
- Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish
- ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch
- Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators
- TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
- INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
- Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
- TurkBench: A Benchmark for Evaluating Turkish Large Language Models
- Introducing TrGLUE and SentiTurca: A Comprehensive Benchmark for Turkish General Language Understanding and Sentiment Analysis
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- Preference Leakage: A Contamination Problem in LLM-as-a-judge
- On Calibration of Modern Neural Networks
- Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
- Overcoming Common Flaws in the Evaluation of Selective Classification Systems
- Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
- ABCD: All Biases Come Disguised
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Croissant: A Metadata Format for ML-Ready Datasets
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering