Is Escalation Worth It? On the Depth of LLM Cascades
This work develops a decision-theoretic framework using optimization and duality to map the cost-quality frontier of LLM cascades. It shows that optimal cascading involves equalizing marginal quality-per-cost across stages, characterized by piecewise concavity and reciprocal shadow prices. Empirically, it validates this by showing that simple pairwise thresholds capture the best performance.