SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models
cs.CL, cs.AI, cs.LG
Submitted: 2026-11-11
Updated: 2026-11-11
Code: https://github.com/holdrain/SO-Arena
Project page: https://lichenlillc.github.io/SO-Arena
Terminology
Sources
- System1-Arena: Which Decisions Can Be Left to System 1?
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- Overview of the TREC 2019 deep learning track
- Overview of the TREC 2020 deep learning track
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Releasing the Inner Jev: A Simple Recipe for Open Decision Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Measuring Massive Multitask Language Understanding
- Do GEO Gains Survive the Live Web? Evaluating In-the-Wild Visibility Gains of Generative Engine Optimization via LIVE-GEO
- ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models
- Jeva: System One Models from First Principles
- TimesJev: Thinking Fast on Time Series with System 1 Sufficiency
- Do Large Language Model Benchmarks Test Reliability?
- LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs
- Constructing Challenging Browser-Use Tasks by Controlled Environment Interventions
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data