Recalibrating a Typed-Decision Model in an Afternoon: A Hands-On Recipe for Laya's Confidence Scores
cs.AI, cs.SE
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/NandhaKishorM/laya
Terminology
Sources
- On Calibration of Modern Neural Networks
- Measuring Calibration in Deep Learning
- Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- Calibration of Pre-trained Transformers
- Selective Classification for Deep Neural Networks
- Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- Verified Uncertainty Calibration
- Revisiting the Calibration of Modern Neural Networks
- Evaluating model calibration in classification
- Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
- CalArena: A Large-Scale Post-Hoc Calibration Benchmark
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
- ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring
- Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?
- Do System One Decisions Add Up? A Study of Probabilistic Coherence
- Benchmarking System One decision models against trained classifiers and language models for automated decision gates
- Jev thinks "I don't know'', but doesn't say it: Introducing Sys1Cal-v1 Dataset for Probability Calibration
- ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty
- VX-One: Typed, Calibrated Decision Heads for Real-Time Perimeter Triage
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents