Exact Calibration and Sharp Risk Geometry for Volume-Sampled Ridge Regression
math.ST, cs.LG, stat.ML, stat.TH
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: 66 pages, 0 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study ridge regression from exactly s distinct rows of a fixed design.
Terminology
Abstract
We study ridge regression from exactly s distinct rows of a fixed design. Responses are fixed, and only the subset is random. The determinant law and selected ridge fit share one positive definite penalty. Established mean identities and exponential-family duality give the unique penalty that matches a prescribed full-data ridge fit in expectation. It exists exactly when s exceeds the target's effective dimension. Our main result concerns centered covariance risk normalized by full-data penalized loss. For balanced signed coordinate replicas, a strict sector inequality gives the sharp risk and all maximizing responses at every budget from the dimension to one below the row count. This holds for any nonzero positive semidefinite query. With the target and query fixed, the maximizing response space is unchanged across these budgets. For general designs, we characterize attainment of a leave-one-out envelope. For existing real equiangular tight frames, flat row query energy characterizes when every nonzero residual response maximizes at two deletions. At three deletions, we give the sharp risk and complete maximizing space for isotropic queries, using unequal triangle weights. The balanced geometry yields a same-sample unbiased ridge--Horvitz--Thompson mixture with lower sharp risk and an exact mean-share improvement boundary. Under full recalibration after feature changes, we prove quadratic regret from searching the complete old maximizing space and a query-uniform bound on the mixture's risk gain. The strongest sector inequalities have exact computer-assisted proofs.
Sources
- Determinantal Point Process Approximation under Positive and Negative Dependence
- Optimal dual frames and dual pairs for probability modelled erasures using weighted average of operator norm and spectral radius
- From DPPs to $k$-DPPs: identifiability analysis via spectral decomposition
- Contact Geometry and Covariance Deficits in Volume-Sampled Least Squares
- Ensemble Kernel Methods, Implicit Regularization and Determinantal Point Processes
Related papers
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
- Bentkus-type asymptotic e-values
- High-Dimensional Asymptotics of Differentially Private PCA
- A proximal gradient algorithm for composite log-concave sampling
- KL Convergence Guarantees for Score diffusion models under minimal data assumptions