Benchmarking System One decision models against trained classifiers and language models for automated decision gates
cs.LG
Submitted: 2026-09-29
Updated: 2026-10-06
Code: https://github.com/pozapas/system-one-models-survey
Terminology
Sources
- AutoMix: Automatically Mixing Language Models
- Distribution-Free, Risk-Controlling Prediction Sets
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- this-that-model-1.0: A typed decision model that decides in 30 ms, for a millionth of a cent
- Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences
- Evaluating and Benchmarking the System One Model Jev
- XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
- Selective Classification for Deep Neural Networks
- Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning
- On Calibration of Modern Neural Networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality
- Evaluating Decision Models for Text Annotation in Computational Social Science
- Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents
- Language Models (Mostly) Know What They Know
- Verified Uncertainty Calibration
- JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks