Conditional Calibration Contracts under Coupled Load: A Branch-Aware Audit of Switched Inference
cs.LG, cs.CL, cs.DC
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 12 pages including references and appendices, ACM sigconf. Historical 40-cell campaign (10 training seeds x 4 coupling levels), with separate three-seed sweeps/ablations. Includes proofs, prospective FIFO mechanism checks, and limitations; prospective contract validation remains open
Code: https://github.com/vectrayx/ca-mhd
License: http://creativecommons.org/licenses/by/4.0/
The gist: Bursty serving changes both queue states and the population selected for each model.
Terminology
Abstract
Bursty serving changes both queue states and the population selected for each model. We study conditional calibration contracts as a specification to be validated, rather than a certified property of an existing router. CA-MHD combines load-coupled distillation, closed-loop replay and a cost-informed Schmitt router in a discrete-event simulator with a hardware service table. Across 40 cells (ten training seeds and four content-load coupling levels), we distinguish the original teacher, frozen wide heads on an adapted trunk, and the narrow student. A common teacher-trajectory audit has median absolute student-teacher deficit 0.06 percentage points, but does not certify the student-selected deployment support. Historical results use a kernel correctness gap, not the published SmoothECE; we separately prove selected-branch composition for a residual functional at a common fixed scale. The deployed wide branch has lower ordinal accuracy than the original teacher in all 40 cells. Same-checkpoint student-only serving has lower historical ordinal gap in 27/40 cells; routing also exposes violation tails up to 17.5 percent. Student p99 latency remains above the 1.5 ms target. Retrospective temporal recalibration improves the ordinal gap in 39/40 cells, but causal delayed-label availability is unverified and no incremental per-mode benefit is detected after multiplicity adjustment. An in-sample CUSUM detects 30/40 checkpoint changes at five historical false alarms, with no prospective false-alarm guarantee. These findings support a branch-aware audit and a scoped negative result, not demonstrated routing superiority. Independent budgets, selected-support validation and net utility under a satisfied SLA remain open.
Sources
- A Unifying Theory of Distance from Calibration
- Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- MatFormer: Nested Transformer for Elastic Inference
- When Your Model Stops Working: Anytime-Valid Calibration Monitoring
- Distilling the Knowledge in a Neural Network
- Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
- SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
- RouteLLM: Learning to Route LLMs with Preference Data
- Calibrate, Then Route: A Measured Study of Learned Request Routing for Disaggregated LLM Serving
- A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis