Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift

arXiv:2610.40331 · math.ST, stat.ME, stat.ML, stat.TH · Submitted 2026-10-29 · Read on arXiv

math.ST, stat.ME, stat.ML, stat.TH

Submitted: 2026-10-29

Updated: 2026-10-29

License: http://creativecommons.org/licenses/by/4.0/

The gist: We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions…

Terminology

Abstract

We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions, sample sizes, and noise variances. Our goal is to combine these sources to estimate the regression function accurately under a target covariate distribution. We propose an oracle weighted neural estimator and establish its minimax optimality up to logarithmic factors through matching upper and lower bounds. The resulting rate is n eff-2p/(2p+K), where p and K denote the effective smoothness and dimension of the regression function, and the effective sample size is n eff = sum from j=1 to J n j over(1+σ ρ,j)σ ε,j squared. Here, n j is the sample size of source j, σ ε,j squared is its noise variance, and σ ρ,j controls the deviation between the covariate distribution of source j and the target covariate distribution. We also develop an adaptive procedure that requires no knowledge of source noise variances or covariate-shift parameters. Under suitable conditions, this procedure attains the oracle minimax rate up to logarithmic factors.

Related papers