Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.
Ada: We're looking at the paper: "Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models".
Title and authors: Ada: So we're looking at the paper called Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models. It sounds pretty dense because it tackles estimating a single linear functional in random design models where everything is dense.
Emmy: Yeah, and that's the core idea: we're not just trying to find one coefficient; we're looking at a specific combination of coefficients, like an exposure coefficient or a contrast among effects.
Kolya: It’s interesting because usually, when the design covariance is unknown and everything is dense, you have to make some big assumptions about sparsity to get any rate at all.
Ada: Exactly. This paper tackles that by looking at the case where neither the regression vector nor the precision matrix has to be sparse. The paper establishes a minimax mean squared rate for this estimation that's n minus alpha of kappa and gamma > one which is strictly better than what you might get under simpler assumptions about diagonal covariance <ref:2610.06875#pg1,minimax mean squared rate for>.
Emmy: It’s about finding the best possible way to estimate that functional when the design matrix itself isn't sparse, which is a big hurdle in high-dimensional statistics.
Kolya: So, we need to see if this rate holds up when we look at the actual numbers and how those parameters kappa and gamma affect it.
The paper's summary: Ada: What the paper actually does is propose an estimator called the Bocca estimator. It uses a complex contour integral involving empirically calibrated resolvents to hit this optimal rate we just talked about.
Emmy: That’s a sophisticated approach because it’s balancing the bias and variance trade-off by using a filter, which they call gK, that has been constructed specifically to achieve that best possible rate.
Kolya: It seems like they are solving this by finding the ideal function g that balances how much noise you get versus how much bias you introduce, and they tie that optimal function directly to some harmonic potential at zero.
Ada: Right. The paper shows that the optimal exponent alpha of kappa and gamma has a closed-form expression using elliptic integrals, specifically showing alpha equals one when gamma is greater than one and it falls between zero and one when both kappa is greater than one and gamma is greater than one.
Emmy: It’s neat that they have this exact mathematical description for the optimal exponent rather than just giving an approximation.
Kolya: That closed form, especially seeing alpha equals one when gamma is greater than one, tells us a lot about how the problem behaves as the number of features p gets much larger relative to the sample size n.
The paper's improvements: Ada: One big improvement they highlight is that their analysis provides a closed-form expression for that optimal exponent alpha in terms of elliptic integrals, which is a major step forward.
Emmy: They also show how the optimal bias variance trade-off function gK is constructed using a specific function defined by some inequalities involving the variance and bias terms, ensuring it hits that rate.
Kolya: The lower bound experiments they set up are pretty rigorous; they construct preliminary Gaussian experiments with covariance eigenvalues between one minus kappa and one, and show that this actually matches the target separation scaled by some fixed positive number c0 over the square root of L.
Ada: That lower bound matching is key because it confirms that n minus alpha of kappa and gamma is indeed the right benchmark for performance evaluation in this setting.
Emmy: It sounds like they are connecting a theoretical rate derived from complex analysis to actual experimental setups, which grounds the theory nicely.
Kolya: So, while the theoretical bound is there, they show how you can actually construct a preliminary experiment that hits that lower bound scaled by some constant factor over the square root of L.
Conclusion: Ada: To wrap up on Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models, the main implication is that we have a concrete estimator, the Bocca estimator, which achieves this optimal rate n minus alpha of kappa and gamma.
Emmy: It confirms that even when you can't assume sparsity in your design matrix or precision matrix, there are specific complex analysis techniques you can use to find the best possible performance bound for these kinds of problems.
Kolya: And from a probabilistic standpoint, it tells us what the true statistical rate is, which is important because it sets the bar for any other estimator we try to build in this dense regime.
Ada: So, we have a rate characterized by those elliptic integrals and a specific way to construct the function gK that balances bias and variance perfectly.
Emmy: It’s a very detailed result, showing how deep structural information about the design covariance spectrum helps determine the estimation speed in these models.
Kolya: I just want to say that seeing that lower bound matched by their construction confirms that this rate is achievable, which is always a nice feeling when you're dealing with these high-dimensional problems.
Junwei Lu
math.ST, stat.TH
Submitted: 2026-09-10
Updated: 2026-11-17
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models studies estimation of a linear functional in random-design linear models when γ = p/n ∈ (1…
Key concepts
- Proportional High-Dimensional Regime ($\gamma = p/n > 1$)
- This regime occurs when the number of parameters ($p$) grows proportionally to the sample size ($n$), specifically $p/n > 1$. In this setting, recovering a low-dimensional functional (like estimating one coefficient) is much harder than recovering the full parameter vector, as both are dense.
- Minimax Mean Squared Rate ($n - \alpha(\kappa, \gamma)$)
- This is the best possible statistical rate achievable for estimating the functional. The exponent $\alpha(\kappa, \gamma)$ depends on model parameters like $\kappa$ (related to eigenvalue bounds) and $\gamma$. This rate is strictly better than what can be achieved under simpler assumptions, such as a diagonal covariance matrix.
- Bocca Estimator
- This is the proposed estimator that achieves the optimal rate. It works by using a complex contour integral with a specific filter function derived from an ideal function $g_K$. This filter balances the trade-off between estimation bias and variance, leading to superior performance compared to simpler methods.
- Elliptic Integrals in Rate Characterization
- The optimal rate exponent $\alpha(\kappa, \gamma)$ is expressed using closed-form elliptic integrals. These special functions mathematically capture the complex geometry of the problem space defined by the model's covariance structure and parameter constraints.
Terminology
Summary
Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models studies estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse The minimax mean squared rate for the linear functional has the order n−α(κ,γ), where α(κ, γ) has an explicit elliptic-integral representation and α(κ, γ) 1 and γ > 1. p. 0 Optimal Linear Functional Estimation for
Proportionally High-Dimensional Linear Models
Junwei Lu∗
Abstract
We study estimation of a linear functional in random-design linear models when γ =
p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and
neither the regression vector nor the precision matrix is assumed sparse. Suppose the
spectrum of the design covariance belongs to [ℓ, L], we establish the minimax mean
squared rate for the linear functional has the order n
−α(κ,γ)
, where the exponent α(κ, γ)
has an explicit elliptic-integral representation and α(κ, γ) < 1 when both κ > 1 and γ >
1.Where the paper says it
Problem Setup and Rate Characterization
The core question addressed is what is the minimax rate of estimation of a fixed linear functional of β under this dense proportional regime when both the regression vector and the precision matrix are dense. The authors establish that for a model family P(n, p, γ, κ), the minimax lower bound is attained in the Gaussian design family G(n, p, γ, κ). p. 2 Consider a sufficiently
large M, so that G(n, p, γ, κ) ⊆ P(n, p, γ, κ). We will show the minimax lower bound is attained
3 8)
Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is
determined by u0(0), the harmonic potential at zero. We can actually have a closed form solution to uδ. 1)
The exponent α(κ, γ) ∈ (0, 1] has a closed-form expression (2. 4) in terms of elliptic integrals. This paper
determines the minimax rate for this problem and characterizes the optimal estimator via the
proportion γ and the design covariance matrix’s condition number κ. Specifically, let ℓ and L be
fixed positive lower and upper bounds of the covariance spectrum, and write κ = L/ℓ ≥ 1. However, the √
n-rate is not achievable under the general dense proportionally
high-dimensional linear models, especially when κ > 1. When both κ and γ diverge, we show that
α(κ, γ) ∼ 4/(
√
κ log γ); see Remark 3. When both κ and γ diverge, we show that
α(κ, γ) ∼ 4/(
√
κ log γ); see Remark 3. 7 for details.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The Bocca Estimator Construction
The paper proposes the Bocca estimator, which combines empirically calibrated resolvents through a complex contour integral to attain the optimal rate. The estimator is defined as βbv(g) = clip[-B0,B0] Re −1/2πiIΓδ 1−g(z)/z Fb(z) dz. p. 5 For every deterministic function g, we define the Bilinear Operator Calibration via Contour
Aggregation (Bocca) estimator as
βbv(g) = clip[−B0,B0] Re
−
1
2πi
I
Γδ
1 − g(z)
z
Fb(z) dz
. 6)
where clip[−B0,B0] A = min{max{A, B0}, −B0} is to ensure the estimator belongs to the model
family that ∥β∥2 ≤ B0. We show the rate of Bocca estimator is
characterized by the bias terms determined by the approximation error of the filter on the covariance spectrum, while its growth on the observation contour controls the variance term. We identify
a reflected elliptic-function as the filter which balances the bias-variance trade-off and attains the
optimal rate. Throughout the paper, we always orient the contour integration counterclockwise. As our parameter
of interest is βv =
R
1 ν( dλ), to approximate it using the right hand side of the above equation, we
need to find a g being small enough on the slit [ℓ, L], while g(0) = 1 removes the extraneous pole
at zero.Where the paper says it
Where the paper says it
Where the paper says it
Bias-Variance Trade-off and Optimality
The bias-variance trade-off of the Bocca estimator is determined by a function g, where the variance term is controlled by sup z∈Γδ g(z)2 and the bias term is determined by sup z∈Λδ g(z)2. The optimal exponent α(κ, γ) is determined by u0(0), the harmonic potential at zero. p. 5 1 in the Appendix for details)
αδ =
uδ(0) − δ
1 − δ
=
α(κ, γ(1 + δ)
2
) − δ
1 − δ
= α(κ, γ) + O(δ). 8)
Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is
determined by u0(0), the harmonic potential at zero. 6)
where clip[−B0,B0] A = min{max{A, B0}, −B0} is to ensure the estimator belongs to the model
family that ∥β∥2 ≤ B0. 1 Bias–variance trade-off
Now we discuss how the function g determines the bias-variance trade-off of the Bocca estimator
and how to construct an ideal g to achieve the optimal rate. 2, gives a constraint on these two values
and gives the closed form of the ideal g to achieve the optimal rate. For sufficiently small δ, every function g holomorphic in {z ∈ Dδ \ [ℓ, L] : δ <
uδ(z) < 1}, continuous on its two boundaries, and normalized by g(0) = 1 satisfies
B(g)
1−αδV (g)
αδ ≥ 1. 17)
Then the Bocca estimator with gK in (3. 15) satisfies
sup
P ∈P(n,p,γ,κ)
EP (βbv(gK) − βv)
2 ≲ n
−α(κ,γ)
.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Information-Theoretic Lower Bound
The information-theoretic lower bound matches the rate of the Bocca estimator in Theorem 4.1. This lower bound is established by constructing preliminary Gaussian experiments with covariance eigenvalues in [κ−1, 1] and showing that the target separation is multiplied by a fixed positive number c0/√L. p. 2 This paper
determines the minimax rate for this problem and characterizes the optimal estimator via the
proportion γ and the design covariance matrix’s condition number κ. Specifically, let ℓ and L be
fixed positive lower and upper bounds of the covariance spectrum, and write κ = L/ℓ ≥ 1. 14) is essentially the two-constants theorem (see Ransford
(1995, Theorem 4. The key insight why u0(0) implies the minimax exponent is because the
extremal case of (3.Where the paper says it
Where the paper says it
Computational Details and Final Rate
The computation of the Bocca estimator involves contour discretization and solving for ζb(µ) using Newton’s method. For the specific choice of g = gK in (3. p. 7 Deterministic kernel values. For the specific choice of g = gK in (3. The two limits agree because
y(t) changes by 2K(1−k
2
) and K is an integer. Thus all kernel evaluations require only real elliptic
integrals and a complex hyperbolic cosine. A common Gaussian transformation converts
the difference in the preliminary noise variances into a difference in the final first coefficients. The
two-hypothesis testing bound then proves the result. 17)
Then the Bocca estimator with gK in (3. 15) satisfies
sup
P ∈P(n,p,γ,κ)
EP (βbv(gK) − βv)
2 ≲ n
−α(κ,γ)
. 10)
Let ueδ be a local harmonic conjugate of uδ, namely uδ + iueδ is holomorphic, normalized by
ueδ(0) = 0, and define the complexification of uδ as
qδ(z) = πK(k
2
)
K(1 − k
2)
uδ(z) + iueδ(z)
. 11)
The above results are derived from the slit-disk form of the classical elliptic rectangle construction
(see Whittaker and Watson (1920, Sections 22. A common Gaussian transformation converts
the difference in the preliminary noise variances into a difference in the final first coefficients. The
two-hypothesis testing bound then proves the result. 13) and the bias-variance trade-off (3. 14) already suggests the
rate n
−αδ which converges to the minimax rate n
−α(κ,γ) as the margin δ shrinks.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
The references listed include works on matrix concentration inequalities, debiasing methods like those of Zhang and Zhang (2014), and spectral analysis of covariance matrices such as Silverstein and Choi (1995). The paper also cites results from Bandeira et al. p. 9 McGraw–Hill, New York. Bandeira, A. In the dense regime when both β and Σ−1 are dense, the source of covariance information
and the inferential target become decisive. Celentano and Montanari (2024) propose correlation
adjusted debiasing, which corrects bias caused by correlated errors in the outcome and covariate
regressions. 10)
Let ueδ be a local harmonic conjugate of uδ, namely uδ + iueδ is holomorphic, normalized by
ueδ(0) = 0, and define the complexification of uδ as
qδ(z) = πK(k
2
)
K(1 − k
2)
uδ(z) + iueδ(z)
. 11)
The above results are derived from the slit-disk form of the classical elliptic rectangle construction
(see Whittaker and Watson (1920, Sections 22. Brailovskaya, T. and van Handel, R. Dicker (2014) develop moment estimators of variance components,
and Verzelen and Gassiat (2018) study adaptive estimation of signal-to-noise ratios with known or
unknown covariance. Kong and Valiant (2018) show how polynomial approximation and moment
estimation can recover prediction-related quantities using fewer observations than features. Acknowledgement. The content of this paper and all of the writing were generated by the author
first. Acknowledgement. The content of this paper and all of the writing were generated by the author
first. The content of this paper and all of the writing were generated by the author
first. 6 was used for literature research, paper polishing, and identifying mathematical
and typographical errors . The content of this paper and all of the writing were generated by the author
first. 6 was used for literature research, paper polishing, and identifying mathematical
and typographical errors .Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 1 ---
Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models Junwei Lu∗ Abstract We study estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ squared = Var(ε). p. 0 , 2022; Sankaranarayanan et al. We study the linear model Y = Xβ +ε,
with unknown design covariance Σ = Cov(X) and unknown noise variance σ
2 = Var(ε). , 2022; Sankaranarayanan et al. We study the linear model Y = Xβ +ε,
with unknown design covariance Σ = Cov(X) and unknown noise variance σ
2 = Var(ε). We study the linear model Y = Xβ +ε,
with unknown design covariance Σ = Cov(X) and unknown noise variance σ
2 = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. We study the linear model Y = Xβ +ε,
with unknown design covariance Σ = Cov(X) and unknown noise variance σ
2 = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . , p} to estimate a
single coefficient βj . Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . , p} to estimate a
single coefficient βj . , p} to estimate a
single coefficient βj . Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and
both the coefficient vector β and the precision matrix Σ−1 may be dense. , p} to estimate a
single coefficient βj . Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and
both the coefficient vector β and the precision matrix Σ−1 may be dense. Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and
both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems. Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and
both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Fix 0 ¯ 0, and set κ = L/l. For integers p > n ≥ 1 and the compatible ratio γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y) ∈ R n×p × R n generated by X = ZΣ(1/2), Y = Xβ + ε, lIp ⪯ Σ ⪯ LIp,∥β∥2 ≤ B0, and σ2 ≤ σ squared. p. 2 For integers p > n ≥ 1 and the compatible ratio
γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y ) ∈ R
n×p × R
n generated by
X = ZΣ
1/2
, Y = Xβ + ε, ℓIp ⪯ Σ ⪯ LIp, ∥β∥2 ≤ B0, and 0 ≤ σ ≤ σ. 1)
Here Σ and β are deterministic parameters, the entries of Z are independent with an identical
distribution satisfying EZ11 = 0, EZ
2
11 = 1, and E|Z11|
r ≤ M for every integer r ≥ 3, and
ε ∼ N(0, σ2
In) is independent of Z. For integers p > n ≥ 1 and the compatible ratio
γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y ) ∈ R
n×p × R
n generated by
X = ZΣ
1/2
, Y = Xβ + ε, ℓIp ⪯ Σ ⪯ LIp, ∥β∥2 ≤ B0, and 0 ≤ σ ≤ σ. 1)
Here Σ and β are deterministic parameters, the entries of Z are independent with an identical
distribution satisfying EZ11 = 0, EZ
2
11 = 1, and E|Z11|
r ≤ M for every integer r ≥ 3, and
ε ∼ N(0, σ2
In) is independent of Z. Define G(n, p, γ, κ) as the subclass in which Z11 ∼ N(0, 1). Consider a sufficiently
large M, so that G(n, p, γ, κ) ⊆ P(n, p, γ, κ). All rate statements in this paper concern sequences of
integers p = p(n) for which
n → ∞ and 1 < lim inf
n→∞
p(n)
n
≤ lim sup
n→∞
p(n)
n
< ∞. 2)
The symbol γ always denotes the current ratio p/n. 2)
The symbol γ always denotes the current ratio p/n. No convergence of p/n to a particular number
is required. No convergence of p/n to a particular number
is required. The number κ = L/ℓ specifies the eigenvalue bounds of the family; it need not be the
condition number of each covariance matrix. The number κ = L/ℓ specifies the eigenvalue bounds of the family; it need not be the
condition number of each covariance matrix. As the diagonal covariance matrix case has been well
studied (Bradic et al. As the diagonal covariance matrix case has been well
studied (Bradic et al. , 2022), in the rest of the paper, we focus on the general dense proportionally
high-dimensional linear models when κ > 1.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
For the eigendecomposition Σ = Pp j=1 λjvjv T j, define ν = Pp j=1(v T v j)(v T j β)δλj. Then the parameter of interest βv = Rν(dλ) and F(µ) = v TΣ(Σ − µIp−1β = Z[l,L] λ - µ ν(dλ). p. 3 3 The Bocca estimator
For the eigendecomposition Σ = Pp
j=1 λjvjv
⊤
j
, define ν =
Pp
j=1(v
⊤vj )(v
⊤
j β)δλj
. Then the parameter of interest βv =
R
ν( dλ) and
F(µ) = v
⊤Σ(Σ − µIp)
−1β =
Z
[ℓ,L]
λ
λ − µ
ν( dλ). Then the parameter of interest βv =
R
ν( dλ) and
F(µ) = v
⊤Σ(Σ − µIp)
−1β =
Z
[ℓ,L]
λ
λ − µ
ν( dλ). 1)
Given some holomorphic function g with g(0) = 1 and some contour Γ enclosing [ℓ, L] ∪ {0}, we
consider the following filter
−
1
2πi
I
Γ
1 − g(z)
z
F(z) dz =
Z
−
1
2πi
I
Γ
1 − g(z)
z
λ
λ − z
dz
ν( dλ) = Z
[ℓ,L]
(1 − g(λ))ν( dλ). As our parameter
of interest is βv =
R
1 ν( dλ), to approximate it using the right hand side of the above equation, we
need to find a g being small enough on the slit [ℓ, L], while g(0) = 1 removes the extraneous pole
at zero. This is the key insight of our Bocca estimator to be proposed formally later.Where the paper says it
Where the paper says it
Where the paper says it
Bias–Variance Trade-off and Optimality
The optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero. The key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero. p. 5 1 in the Appendix for details)
αδ =
uδ(0) − δ
1 − δ
=
α(κ, γ(1 + δ)
2
) − δ
1 − δ
= α(κ, γ) + O(δ). 8)
Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is
determined by u0(0), the harmonic potential at zero. Define Λδ = {z ∈ Dδ \ [ℓ, L] : uδ(z) = δ} and
B(g) = sup
z∈Λδ
|g(z)|
2
, V (g) = sup
z∈Γδ
|g(z)|
2
. 12)
Then, for all sufficiently large n, we have
sup
P ∈P(n,p,γ,κ)
EP (βbv(g) − βv)
2 ≲ B(g) + V (g)
n
. The next theorem, whose proof is deferred in Appendix B. 2, gives a constraint on these two values
and gives the closed form of the ideal g to achieve the optimal rate.Where the paper says it
Where the paper says it
Where the paper says it
Lower Bound Experiments
The scalar minimax lower bound is established by constructing preliminary Gaussian experiments with covariance eigenvalues in [κ−1, 1] and showing that the target separation is multiplied by a fixed positive number c0/√L. This testing argument proves (4. p. 8 A common Gaussian transformation converts
the difference in the preliminary noise variances into a difference in the final first coefficients. The
two-hypothesis testing bound then proves the result. Endpoint weights ensure the final covariance bounds; a weighted spectral integral controls the
expected conditional Kullback–Leibler divergence. A common Gaussian transformation converts
the difference in the preliminary noise variances into a difference in the final first coefficients.Where the paper says it
Where the paper says it
Supporting Analytic Estimates
The appendix provides formulas for uδ and qδ used in Section 3.1, the geometry of the level curve Λδ, and the contour formula (3. p. 6 The right-hand side of (3. 13) explicitly shows how the bias-variance trade-off depends on g: the
variance term is determined by the maximum value of g on the contour Γδ and the bias term is
determined by the maximum value of g on Λδ which will shrink to the slit [ℓ, L] as δ goes to zero. We propose the Bilinear Operator Calibration via Contour Aggregation (Bocca) estimator,
which combines empirically calibrated resolvents through a complex contour integral. The calibration is motivated by the companion Stieltjes equation (Silverstein and Choi, 1995) and anisotropic
local law (Knowles and Yin, 2017) for sample covariance matrices.Where the paper says it
Where the paper says it
--- Page 2 ---
Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models Junwei Lu∗ Abstract We study estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ squared = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ 1,..., p to estimate a single coefficient βj. Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Optimal rate determination for estimation of a fixed linear functional in dense proportional high-dimensional linear models when both regression vector and precision matrix are dense allows AI systems to achieve an optimal minimax mean squared rate of n minus alpha(kappa, gamma), which is strictly better than the root n consistency achievable under diagonal covariance.
-
The Bocca estimator, utilizing a complex contour integral combining empirically calibrated resolvents through a filter, enables AI systems to attain this optimal rate by balancing bias and variance trade-off through an ideal function gK that satisfies B(gK) over V(g).
-
AI systems can implement the Bocca estimator computation by discretizing the contour, computing the eigendecomposition of Sb to find mb(z), and solving for zeta b(mu) using Newton's method to estimate Fb(mu).
-
The analysis provides a closed-form expression for the optimal exponent alpha(kappa, gamma) in terms of elliptic integrals, specifically showing that α (1, gamma) equals 1 when gamma > 1 and α (kappa, gamma) is between 0 and 1 when kappa > 1 and gamma > 1.
-
The system can utilize the derived harmonic potential u delta to determine the optimal bias-variance trade-off by identifying u0(0), which dictates the minimax exponent alpha(kappa, gamma).
-
A specific estimator, Bocca with gK defined by cosh (Kq delta) and hyperbolic cosine functions, is proposed to achieve the optimal rate n minus alpha(kappa, gamma), where K is chosen to balance the bias and variance terms as specified in Theorem 3.6.
-
The lower bound established by Theorem 4.1 shows that for any estimator of a fixed linear functional βv, the minimax rate is at least n minus alpha(kappa, gamma), providing a benchmark for performance evaluation.
-
The system can construct preliminary Gaussian experiments with covariance eigenvalues in [kappa minus 1, 1] and scale them to match the problem family P(n, p, gamma, kappa), verifying that these experiments achieve the lower bound of n minus alpha(kappa, gamma)/2.
-
The framework allows for the construction of supported priors Q0 and Q1 whose data laws satisfy TV(P0, P1) less than or equal to 1/3 while ensuring b1 minus b0 is at least ce minus Kq(0) slash n minus alpha (kappa, p/n)/2.
-
The system can generate the final Gaussian regression parameters using a mixing distribution Pi(V) and conditional regressions where the noise variance sigma 2 j is bounded within the interval four varj s0 over varj plus s0, satisfying 4s0 over three less than or equal to sigma 2 j less than or equal to two s0.
-
The system can verify that the total variation distance between data laws PeC j and Pej is at most C tau (ne minus two Kh plus K squared slash n) one half plus Ce minus cn u squared, which is shown to be at most 1/4 under condition F(1).
Abstract
We study estimation of a linear functional in random-design linear models when γ=p/n in (1, infinity) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. Suppose the spectrum of the design covariance belongs to [,L], and let κ=L/. We establish that the minimax mean squared error for the linear functional is of order n-α(κ,γ), where the exponent α(κ,γ) has an explicit elliptic-integral representation and α(κ,γ) < 1 when both κ> 1 and γ> 1. We propose the Bocca estimator, which combines empirically calibrated resolvents through a complex contour integral to attain the optimal rate. We also establish asymptotic normality of the Bocca estimator and develop a variance estimator for valid inference.
Sources
- Challenges of the inconsistency regime: Novel debiasing methods for missing data models
- Method-of-Moments Inference for GLMs and Doubly Robust Functionals under Proportional Asymptotics
- Spectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression
- High-Dimensional Statistics: Reflections on Progress and Open Problems
- Characterizing Finite-Dimensional Posterior Marginals in High-Dimensional GLMs via Leave-One-Out
- Moment-Based Adjustments of Statistical Inference in High-Dimensional Generalized Linear Models
Related papers
- Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators
- Nonparametric Regression on Fixed-Cardinality Subsets:Minimax Risk and Random-Design Effects
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Semiparametric Efficient Inference under Non-Informative Complex Survey Designs
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
- Bentkus-type asymptotic e-values