Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift
math.ST, stat.ME, stat.ML, stat.TH
Submitted: 2026-10-29
Updated: 2026-10-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions…
Terminology
Abstract
We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions, sample sizes, and noise variances. Our goal is to combine these sources to estimate the regression function accurately under a target covariate distribution. We propose an oracle weighted neural estimator and establish its minimax optimality up to logarithmic factors through matching upper and lower bounds. The resulting rate is n eff-2p/(2p+K), where p and K denote the effective smoothness and dimension of the regression function, and the effective sample size is n eff = sum from j=1 to J n j over(1+σ ρ,j)σ ε,j squared. Here, n j is the sample size of source j, σ ε,j squared is its noise variance, and σ ρ,j controls the deviation between the covariate distribution of source j and the target covariate distribution. We also develop an adaptive procedure that requires no knowledge of source noise variances or covariate-shift parameters. Under suitable conditions, this procedure attains the oracle minimax rate up to logarithmic factors.
Related papers
- Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
- Bentkus-type asymptotic e-values
- Canonical Layer Inference for Deep Gaussian Processes via Symmetry-Breaking Augmentation
- High-Dimensional Asymptotics of Differentially Private PCA