Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models

arXiv:2610.06875 · math.ST, stat.TH · Submitted 2026-09-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: We're looking at the paper: "Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models".

Title and authors: Ada: So we're looking at the paper called Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models. It sounds pretty dense because it tackles estimating a single linear functional in random design models where everything is dense.

Emmy: Yeah, and that's the core idea: we're not just trying to find one coefficient; we're looking at a specific combination of coefficients, like an exposure coefficient or a contrast among effects.

Kolya: It’s interesting because usually, when the design covariance is unknown and everything is dense, you have to make some big assumptions about sparsity to get any rate at all.

Ada: Exactly. This paper tackles that by looking at the case where neither the regression vector nor the precision matrix has to be sparse. The paper establishes a minimax mean squared rate for this estimation that's n minus alpha of kappa and gamma > one which is strictly better than what you might get under simpler assumptions about diagonal covariance <ref:2610.06875#pg1,minimax mean squared rate for>.

Emmy: It’s about finding the best possible way to estimate that functional when the design matrix itself isn't sparse, which is a big hurdle in high-dimensional statistics.

Kolya: So, we need to see if this rate holds up when we look at the actual numbers and how those parameters kappa and gamma affect it.

The paper's summary: Ada: What the paper actually does is propose an estimator called the Bocca estimator. It uses a complex contour integral involving empirically calibrated resolvents to hit this optimal rate we just talked about.

Emmy: That’s a sophisticated approach because it’s balancing the bias and variance trade-off by using a filter, which they call gK, that has been constructed specifically to achieve that best possible rate.

Kolya: It seems like they are solving this by finding the ideal function g that balances how much noise you get versus how much bias you introduce, and they tie that optimal function directly to some harmonic potential at zero.

Ada: Right. The paper shows that the optimal exponent alpha of kappa and gamma has a closed-form expression using elliptic integrals, specifically showing alpha equals one when gamma is greater than one and it falls between zero and one when both kappa is greater than one and gamma is greater than one.

Emmy: It’s neat that they have this exact mathematical description for the optimal exponent rather than just giving an approximation.

Kolya: That closed form, especially seeing alpha equals one when gamma is greater than one, tells us a lot about how the problem behaves as the number of features p gets much larger relative to the sample size n.

The paper's improvements: Ada: One big improvement they highlight is that their analysis provides a closed-form expression for that optimal exponent alpha in terms of elliptic integrals, which is a major step forward.

Emmy: They also show how the optimal bias variance trade-off function gK is constructed using a specific function defined by some inequalities involving the variance and bias terms, ensuring it hits that rate.

Kolya: The lower bound experiments they set up are pretty rigorous; they construct preliminary Gaussian experiments with covariance eigenvalues between one minus kappa and one, and show that this actually matches the target separation scaled by some fixed positive number c0 over the square root of L.

Ada: That lower bound matching is key because it confirms that n minus alpha of kappa and gamma is indeed the right benchmark for performance evaluation in this setting.

Emmy: It sounds like they are connecting a theoretical rate derived from complex analysis to actual experimental setups, which grounds the theory nicely.

Kolya: So, while the theoretical bound is there, they show how you can actually construct a preliminary experiment that hits that lower bound scaled by some constant factor over the square root of L.

Conclusion: Ada: To wrap up on Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models, the main implication is that we have a concrete estimator, the Bocca estimator, which achieves this optimal rate n minus alpha of kappa and gamma.

Emmy: It confirms that even when you can't assume sparsity in your design matrix or precision matrix, there are specific complex analysis techniques you can use to find the best possible performance bound for these kinds of problems.

Kolya: And from a probabilistic standpoint, it tells us what the true statistical rate is, which is important because it sets the bar for any other estimator we try to build in this dense regime.

Ada: So, we have a rate characterized by those elliptic integrals and a specific way to construct the function gK that balances bias and variance perfectly.

Emmy: It’s a very detailed result, showing how deep structural information about the design covariance spectrum helps determine the estimation speed in these models.

Kolya: I just want to say that seeing that lower bound matched by their construction confirms that this rate is achievable, which is always a nice feeling when you're dealing with these high-dimensional problems.

Junwei Lu

math.ST, stat.TH

Submitted: 2026-09-10

Updated: 2026-11-17

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models studies estimation of a linear functional in random-design linear models when γ = p/n ∈ (1…

Key concepts

Proportional High-Dimensional Regime ($\gamma = p/n > 1$)
This regime occurs when the number of parameters ($p$) grows proportionally to the sample size ($n$), specifically $p/n > 1$. In this setting, recovering a low-dimensional functional (like estimating one coefficient) is much harder than recovering the full parameter vector, as both are dense.
Minimax Mean Squared Rate ($n - \alpha(\kappa, \gamma)$)
This is the best possible statistical rate achievable for estimating the functional. The exponent $\alpha(\kappa, \gamma)$ depends on model parameters like $\kappa$ (related to eigenvalue bounds) and $\gamma$. This rate is strictly better than what can be achieved under simpler assumptions, such as a diagonal covariance matrix.
Bocca Estimator
This is the proposed estimator that achieves the optimal rate. It works by using a complex contour integral with a specific filter function derived from an ideal function $g_K$. This filter balances the trade-off between estimation bias and variance, leading to superior performance compared to simpler methods.
Elliptic Integrals in Rate Characterization
The optimal rate exponent $\alpha(\kappa, \gamma)$ is expressed using closed-form elliptic integrals. These special functions mathematically capture the complex geometry of the problem space defined by the model's covariance structure and parameter constraints.

Terminology

Summary

Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models studies estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse The minimax mean squared rate for the linear functional has the order n−α(κ,γ), where α(κ, γ) has an explicit elliptic-integral representation and α(κ, γ) 1 and γ > 1. p. 0

Where the paper says it

Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models Junwei Lu∗ Abstract We study estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. Suppose the spectrum of the design covariance belongs to [ℓ, L], we establish the minimax mean squared rate for the linear functional has the order n −α(κ,γ) , where the exponent α(κ, γ) has an explicit elliptic-integral representation and α(κ, γ) < 1 when both κ > 1 and γ > 1.

Problem Setup and Rate Characterization

The core question addressed is what is the minimax rate of estimation of a fixed linear functional of β under this dense proportional regime when both the regression vector and the precision matrix are dense. The authors establish that for a model family P(n, p, γ, κ), the minimax lower bound is attained in the Gaussian design family G(n, p, γ, κ). p. 2

Where the paper says it

Consider a sufficiently large M, so that G(n, p, γ, κ) ⊆ P(n, p, γ, κ). We will show the minimax lower bound is attained 3

The rate of estimation is characterized by the exponent α(κ, γ) which has a closed-form expression (2. p. 5
Where the paper says it

8) Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero. We can actually have a closed form solution to uδ.

4) in terms of elliptic integrals. p. 2
Where the paper says it

1) The exponent α(κ, γ) ∈ (0, 1] has a closed-form expression (2. 4) in terms of elliptic integrals.

Specifically, α(1, γ) = 1 when γ > 1 and α(κ, γ) ∈ (0, 1) when both κ > 1 and γ > 1. p. 2
Where the paper says it

This paper determines the minimax rate for this problem and characterizes the optimal estimator via the proportion γ and the design covariance matrix’s condition number κ. Specifically, let ℓ and L be fixed positive lower and upper bounds of the covariance spectrum, and write κ = L/ℓ ≥ 1.

The asymptotic behavior of the exponent is shown to be α(κ, γ) ∼ 4/(√κ log γ); see Remark 3. p. 2
Where the paper says it

However, the √ n-rate is not achievable under the general dense proportionally high-dimensional linear models, especially when κ > 1. When both κ and γ diverge, we show that α(κ, γ) ∼ 4/( √ κ log γ); see Remark 3.

7 for details. p. 2
Where the paper says it

When both κ and γ diverge, we show that α(κ, γ) ∼ 4/( √ κ log γ); see Remark 3. 7 for details.

The Bocca Estimator Construction

The paper proposes the Bocca estimator, which combines empirically calibrated resolvents through a complex contour integral to attain the optimal rate. The estimator is defined as βbv(g) = clip[-B0,B0] Re −1/2πiIΓδ 1−g(z)/z Fb(z) dz. p. 5

Where the paper says it

For every deterministic function g, we define the Bilinear Operator Calibration via Contour Aggregation (Bocca) estimator as βbv(g) = clip[−B0,B0] Re − 1 2πi I Γδ 1 − g(z) z Fb(z) dz . 6) where clip[−B0,B0] A = min{max{A, B0}, −B0} is to ensure the estimator belongs to the model family that ∥β∥2 ≤ B0.

This estimator balances the bias-variance trade-off by identifying a reflected elliptic-function as the filter which attains the optimal rate. p. 2
Where the paper says it

We show the rate of Bocca estimator is characterized by the bias terms determined by the approximation error of the filter on the covariance spectrum, while its growth on the observation contour controls the variance term. We identify a reflected elliptic-function as the filter which balances the bias-variance trade-off and attains the optimal rate.

The construction involves solving for a function g being small enough on the slit [l, L], while g(0) = 1 removes an extraneous pole at zero. p. 3
Where the paper says it

Throughout the paper, we always orient the contour integration counterclockwise. As our parameter of interest is βv = R 1 ν( dλ), to approximate it using the right hand side of the above equation, we need to find a g being small enough on the slit [ℓ, L], while g(0) = 1 removes the extraneous pole at zero.

Bias-Variance Trade-off and Optimality

The bias-variance trade-off of the Bocca estimator is determined by a function g, where the variance term is controlled by sup z∈Γδ g(z)2 and the bias term is determined by sup z∈Λδ g(z)2. The optimal exponent α(κ, γ) is determined by u0(0), the harmonic potential at zero. p. 5

Where the paper says it

1 in the Appendix for details) αδ = uδ(0) − δ 1 − δ = α(κ, γ(1 + δ) 2 ) − δ 1 − δ = α(κ, γ) + O(δ). 8) Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero.

The ideal g to achieve the optimal rate is constructed using a function gK in (3. p. 5
Where the paper says it

6) where clip[−B0,B0] A = min{max{A, B0}, −B0} is to ensure the estimator belongs to the model family that ∥β∥2 ≤ B0. 1 Bias–variance trade-off Now we discuss how the function g determines the bias-variance trade-off of the Bocca estimator and how to construct an ideal g to achieve the optimal rate.

15), which satisfies B(gK) / (1−αδV(g)) αδ ≥ 1. p. 6
Where the paper says it

2, gives a constraint on these two values and gives the closed form of the ideal g to achieve the optimal rate. For sufficiently small δ, every function g holomorphic in {z ∈ Dδ \ [ℓ, L] : δ < uδ(z) < 1}, continuous on its two boundaries, and normalized by g(0) = 1 satisfies B(g) 1−αδV (g) αδ ≥ 1.

This leads to the final estimator satisfying sup P ∈P(n,p,γ,κ) EP (βbv(gK) − βv)2 ≲ n−αδ. p. 6
Where the paper says it

17) Then the Bocca estimator with gK in (3. 15) satisfies sup P ∈P(n,p,γ,κ) EP (βbv(gK) − βv) 2 ≲ n −α(κ,γ) .

Information-Theoretic Lower Bound

The information-theoretic lower bound matches the rate of the Bocca estimator in Theorem 4.1. This lower bound is established by constructing preliminary Gaussian experiments with covariance eigenvalues in [κ−1, 1] and showing that the target separation is multiplied by a fixed positive number c0/√L. p. 2

Where the paper says it

This paper determines the minimax rate for this problem and characterizes the optimal estimator via the proportion γ and the design covariance matrix’s condition number κ. Specifically, let ℓ and L be fixed positive lower and upper bounds of the covariance spectrum, and write κ = L/ℓ ≥ 1.

This comparison proves the two scalar minimax comparisons stated in the theorem. p. 6
Where the paper says it

14) is essentially the two-constants theorem (see Ransford (1995, Theorem 4. The key insight why u0(0) implies the minimax exponent is because the extremal case of (3.

Computational Details and Final Rate

The computation of the Bocca estimator involves contour discretization and solving for ζb(µ) using Newton’s method. For the specific choice of g = gK in (3. p. 7

Where the paper says it

Deterministic kernel values. For the specific choice of g = gK in (3.

15), the kernel evaluations require only real elliptic integrals and a complex hyperbolic cosine. p. 8
Where the paper says it

The two limits agree because y(t) changes by 2K(1−k 2 ) and K is an integer. Thus all kernel evaluations require only real elliptic integrals and a complex hyperbolic cosine.

The final theorem proves that the Bocca estimator with gK in (3. p. 8
Where the paper says it

A common Gaussian transformation converts the difference in the preliminary noise variances into a difference in the final first coefficients. The two-hypothesis testing bound then proves the result.

15) satisfies sup P ∈P(n,p,γ,κ) EP (βbv(gK) − βv)2 ≲ n−α(κ,γ). p. 6
Where the paper says it

17) Then the Bocca estimator with gK in (3. 15) satisfies sup P ∈P(n,p,γ,κ) EP (βbv(gK) − βv) 2 ≲ n −α(κ,γ) .

The results on the elliptic integral formulas prove the exact identity in (3. p. 5
Where the paper says it

10) Let ueδ be a local harmonic conjugate of uδ, namely uδ + iueδ is holomorphic, normalized by ueδ(0) = 0, and define the complexification of uδ as qδ(z) = πK(k 2 ) K(1 − k 2) uδ(z) + iueδ(z) . 11) The above results are derived from the slit-disk form of the classical elliptic rectangle construction (see Whittaker and Watson (1920, Sections 22.

8). The final result is that log(16γ)/ log γ → 1, which proves (3. p. 8
Where the paper says it

A common Gaussian transformation converts the difference in the preliminary noise variances into a difference in the final first coefficients. The two-hypothesis testing bound then proves the result.

19). The paper concludes by showing the rate n−αδ converges to the minimax rate n−α(κ,γ) as the margin δ shrinks. p. 6
Where the paper says it

13) and the bias-variance trade-off (3. 14) already suggests the rate n −αδ which converges to the minimax rate n −α(κ,γ) as the margin δ shrinks.

References

The references listed include works on matrix concentration inequalities, debiasing methods like those of Zhang and Zhang (2014), and spectral analysis of covariance matrices such as Silverstein and Choi (1995). The paper also cites results from Bandeira et al. p. 9

Where the paper says it

McGraw–Hill, New York. Bandeira, A.

(2023) regarding correlation adjusted debiasing. p. 1
Where the paper says it

In the dense regime when both β and Σ−1 are dense, the source of covariance information and the inferential target become decisive. Celentano and Montanari (2024) propose correlation adjusted debiasing, which corrects bias caused by correlated errors in the outcome and covariate regressions.

The work relies on complex analysis tools from Ahlfors (1979) and Whittaker and Watson (1920) for the elliptic integral identities. p. 5
Where the paper says it

10) Let ueδ be a local harmonic conjugate of uδ, namely uδ + iueδ is holomorphic, normalized by ueδ(0) = 0, and define the complexification of uδ as qδ(z) = πK(k 2 ) K(1 − k 2) uδ(z) + iueδ(z) . 11) The above results are derived from the slit-disk form of the classical elliptic rectangle construction (see Whittaker and Watson (1920, Sections 22.

The final section includes detailed proofs involving moment expansions, such as those by Boucheron et al. (2005), and spectral comparison theorems from Brailovskaya and van Handel (2024). p. 9
Where the paper says it

Brailovskaya, T. and van Handel, R.

The paper also references results on Gaussian pair partitions from Kong and Valiant (2018). p. 1
Where the paper says it

Dicker (2014) develop moment estimators of variance components, and Verzelen and Gassiat (2018) study adaptive estimation of signal-to-noise ratios with known or unknown covariance. Kong and Valiant (2018) show how polynomial approximation and moment estimation can recover prediction-related quantities using fewer observations than features.

The acknowledgement section notes that the content was generated by the author first, with ChatGPT-5. p. 8
Where the paper says it

Acknowledgement. The content of this paper and all of the writing were generated by the author first.

p. 8
Where the paper says it

Acknowledgement. The content of this paper and all of the writing were generated by the author first.

6 used for literature research, paper polishing, and identifying mathematical and typographical errors. p. 8
Where the paper says it

The content of this paper and all of the writing were generated by the author first. 6 was used for literature research, paper polishing, and identifying mathematical and typographical errors .

p. 8
Where the paper says it

The content of this paper and all of the writing were generated by the author first. 6 was used for literature research, paper polishing, and identifying mathematical and typographical errors .

--- Page 1 ---

Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models Junwei Lu∗ Abstract We study estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ squared = Var(ε). p. 0

Where the paper says it

, 2022; Sankaranarayanan et al. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ 2 = Var(ε).

p. 0
Where the paper says it

, 2022; Sankaranarayanan et al. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ 2 = Var(ε).

The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. p. 0
Where the paper says it

We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ 2 = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector.

p. 0
Where the paper says it

We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ 2 = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector.

Most of the time, we consider the canonical vector v = ej for some j ∈ 1,. p. 0
Where the paper says it

The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ {1, .

p. 0
Where the paper says it

The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ {1, .

.., p to estimate a single coefficient βj. p. 0
Where the paper says it

Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . , p} to estimate a single coefficient βj .

p. 0
Where the paper says it

Most of the time, we consider the canonical vector v = ej for some j ∈ {1, . , p} to estimate a single coefficient βj .

Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense. p. 0
Where the paper says it

, p} to estimate a single coefficient βj . Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense.

p. 0
Where the paper says it

, p} to estimate a single coefficient βj . Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense.

In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems. p. 0
Where the paper says it

Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems.

p. 0
Where the paper says it

Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems.

Fix 0 ¯ 0, and set κ = L/l. For integers p > n ≥ 1 and the compatible ratio γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y) ∈ R n×p × R n generated by X = ZΣ(1/2), Y = Xβ + ε, lIp ⪯ Σ ⪯ LIp,∥β∥2 ≤ B0, and σ2 ≤ σ squared. p. 2

Where the paper says it

For integers p > n ≥ 1 and the compatible ratio γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y ) ∈ R n×p × R n generated by X = ZΣ 1/2 , Y = Xβ + ε, ℓIp ⪯ Σ ⪯ LIp, ∥β∥2 ≤ B0, and 0 ≤ σ ≤ σ. 1) Here Σ and β are deterministic parameters, the entries of Z are independent with an identical distribution satisfying EZ11 = 0, EZ 2 11 = 1, and E|Z11| r ≤ M for every integer r ≥ 3, and ε ∼ N(0, σ2 In) is independent of Z.

Here Σ and β are deterministic parameters, the entries of Z are independent with an identical distribution satisfying EZ11 = 0, EZ T11 = 1, and EZ r 1r ≤ M for every integer r ≥ 3, and ε ∼ N(0, σ 2/n) is independent of Z. p. 2
Where the paper says it

For integers p > n ≥ 1 and the compatible ratio γ = p/n, let P(n, p, γ, κ) be the family of joint laws of (X, Y ) ∈ R n×p × R n generated by X = ZΣ 1/2 , Y = Xβ + ε, ℓIp ⪯ Σ ⪯ LIp, ∥β∥2 ≤ B0, and 0 ≤ σ ≤ σ. 1) Here Σ and β are deterministic parameters, the entries of Z are independent with an identical distribution satisfying EZ11 = 0, EZ 2 11 = 1, and E|Z11| r ≤ M for every integer r ≥ 3, and ε ∼ N(0, σ2 In) is independent of Z.

We consider a sufficiently large M, so that G(n, p, γ, κ) ⊆ P(n, p, γ, κ). p. 2
Where the paper says it

Define G(n, p, γ, κ) as the subclass in which Z11 ∼ N(0, 1). Consider a sufficiently large M, so that G(n, p, γ, κ) ⊆ P(n, p, γ, κ).

The symbol γ always denotes the current ratio p/n. p. 3
Where the paper says it

All rate statements in this paper concern sequences of integers p = p(n) for which n → ∞ and 1 < lim inf n→∞ p(n) n ≤ lim sup n→∞ p(n) n < ∞. 2) The symbol γ always denotes the current ratio p/n.

No convergence of p/n to a particular number is required. p. 3
Where the paper says it

2) The symbol γ always denotes the current ratio p/n. No convergence of p/n to a particular number is required.

The number κ = L/l specifies the eigenvalue bounds of the family; it need not be the condition number of each covariance matrix. p. 3
Where the paper says it

No convergence of p/n to a particular number is required. The number κ = L/ℓ specifies the eigenvalue bounds of the family; it need not be the condition number of each covariance matrix.

As the diagonal covariance matrix case has been well studied (Bradic et al. p. 3
Where the paper says it

The number κ = L/ℓ specifies the eigenvalue bounds of the family; it need not be the condition number of each covariance matrix. As the diagonal covariance matrix case has been well studied (Bradic et al.

, 2022), in the rest of the paper, we focus on the general dense proportionally high-dimensional linear models when κ > 1. p. 3
Where the paper says it

As the diagonal covariance matrix case has been well studied (Bradic et al. , 2022), in the rest of the paper, we focus on the general dense proportionally high-dimensional linear models when κ > 1.

For the eigendecomposition Σ = Pp j=1 λjvjv T j, define ν = Pp j=1(v T v j)(v T j β)δλj. Then the parameter of interest βv = Rν(dλ) and F(µ) = v TΣ(Σ − µIp−1β = Z[l,L] λ - µ ν(dλ). p. 3

Where the paper says it

3 The Bocca estimator For the eigendecomposition Σ = Pp j=1 λjvjv ⊤ j , define ν = Pp j=1(v ⊤vj )(v ⊤ j β)δλj . Then the parameter of interest βv = R ν( dλ) and F(µ) = v ⊤Σ(Σ − µIp) −1β = Z [ℓ,L] λ λ − µ ν( dλ).

Given some holomorphic function g with g(0) = 1 and some contour Γ enclosing [l, L] ∪ 0, we consider the following filter −1/2πiIΓ1−g(z)/z F(z) dz = Z − 1/2πiIΓλ−g(λ)ν(dλ). p. 3
Where the paper says it

Then the parameter of interest βv = R ν( dλ) and F(µ) = v ⊤Σ(Σ − µIp) −1β = Z [ℓ,L] λ λ − µ ν( dλ). 1) Given some holomorphic function g with g(0) = 1 and some contour Γ enclosing [ℓ, L] ∪ {0}, we consider the following filter − 1 2πi I Γ 1 − g(z) z F(z) dz = Z − 1 2πi I Γ 1 − g(z) z λ λ − z dz ν( dλ) = Z [ℓ,L] (1 − g(λ))ν( dλ).

This is the key insight of our Bocca estimator to be proposed formally later. p. 3
Where the paper says it

As our parameter of interest is βv = R 1 ν( dλ), to approximate it using the right hand side of the above equation, we need to find a g being small enough on the slit [ℓ, L], while g(0) = 1 removes the extraneous pole at zero. This is the key insight of our Bocca estimator to be proposed formally later.

Bias–Variance Trade-off and Optimality

The optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero. The key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero. p. 5

Where the paper says it

1 in the Appendix for details) αδ = uδ(0) − δ 1 − δ = α(κ, γ(1 + δ) 2 ) − δ 1 − δ = α(κ, γ) + O(δ). 8) Therefore, the key insight is that the optimal exponent α(κ, γ) for the estimation rate of βv is determined by u0(0), the harmonic potential at zero.

Theorem 3.3 shows that for all sufficiently large n, we have sup P ∈P(n,p,γ,κ) EP (βbv(gK) − βv)2 ≲ B(g) + V(g)/n. p. 5
Where the paper says it

Define Λδ = {z ∈ Dδ \ [ℓ, L] : uδ(z) = δ} and B(g) = sup z∈Λδ |g(z)| 2 , V (g) = sup z∈Γδ |g(z)| 2 . 12) Then, for all sufficiently large n, we have sup P ∈P(n,p,γ,κ) EP (βbv(g) − βv) 2 ≲ B(g) + V (g) n .

The next theorem gives a constraint on these two values and gives the closed form of the ideal g to achieve the optimal rate. p. 6
Where the paper says it

The next theorem, whose proof is deferred in Appendix B. 2, gives a constraint on these two values and gives the closed form of the ideal g to achieve the optimal rate.

Lower Bound Experiments

The scalar minimax lower bound is established by constructing preliminary Gaussian experiments with covariance eigenvalues in [κ−1, 1] and showing that the target separation is multiplied by a fixed positive number c0/√L. This testing argument proves (4. p. 8

Where the paper says it

A common Gaussian transformation converts the difference in the preliminary noise variances into a difference in the final first coefficients. The two-hypothesis testing bound then proves the result.

1) on G(n, p, γ, κ). The construction involves preliminary experiments with only p−1 covariates and a final experiment with dimension p. p. 8
Where the paper says it

Endpoint weights ensure the final covariance bounds; a weighted spectral integral controls the expected conditional Kullback–Leibler divergence. A common Gaussian transformation converts the difference in the preliminary noise variances into a difference in the final first coefficients.

Supporting Analytic Estimates

The appendix provides formulas for uδ and qδ used in Section 3.1, the geometry of the level curve Λδ, and the contour formula (3. p. 6

Where the paper says it

The right-hand side of (3. 13) explicitly shows how the bias-variance trade-off depends on g: the variance term is determined by the maximum value of g on the contour Γδ and the bias term is determined by the maximum value of g on Λδ which will shrink to the slit [ℓ, L] as δ goes to zero.

22). The results on the elliptic integral formulas prove that log(16γ)/ log γ → 1, which proves (3.19). The paper also references results from Silverstein and Choi (1995) regarding the Stieltjes transform and its properties. p. 2
Where the paper says it

We propose the Bilinear Operator Calibration via Contour Aggregation (Bocca) estimator, which combines empirically calibrated resolvents through a complex contour integral. The calibration is motivated by the companion Stieltjes equation (Silverstein and Choi, 1995) and anisotropic local law (Knowles and Yin, 2017) for sample covariance matrices.

The final section includes detailed proofs involving moment expansions, such as those by Yang (2024). The acknowledgement section notes that the content was generated by the author first, with ChatGPT-5.6 used for literature research, paper polishing, and identifying mathematical and typographical errors.

--- Page 2 ---

Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models Junwei Lu∗ Abstract We study estimation of a linear functional in random-design linear models when γ = p/n ∈ (1, ∞) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. We study the linear model Y = Xβ +ε, with unknown design covariance Σ = Cov(X) and unknown noise variance σ squared = Var(ε). The parameter of interest is the scalar functional βv = ⟨v, β⟩, where v is a known deterministic unit vector. Most of the time, we consider the canonical vector v = ej for some j ∈ 1,..., p to estimate a single coefficient βj. Our focus is the dense proportional regime in which γ = p/n ∈ (1, ∞) and both the coefficient vector β and the precision matrix Σ−1 may be dense. In this regime, recovering a low-dimensional functional and recovering the full vector are fundamentally different problems.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Optimal rate determination for estimation of a fixed linear functional in dense proportional high-dimensional linear models when both regression vector and precision matrix are dense allows AI systems to achieve an optimal minimax mean squared rate of n minus alpha(kappa, gamma), which is strictly better than the root n consistency achievable under diagonal covariance.

  2. The Bocca estimator, utilizing a complex contour integral combining empirically calibrated resolvents through a filter, enables AI systems to attain this optimal rate by balancing bias and variance trade-off through an ideal function gK that satisfies B(gK) over V(g).

  3. AI systems can implement the Bocca estimator computation by discretizing the contour, computing the eigendecomposition of Sb to find mb(z), and solving for zeta b(mu) using Newton's method to estimate Fb(mu).

  4. The analysis provides a closed-form expression for the optimal exponent alpha(kappa, gamma) in terms of elliptic integrals, specifically showing that α (1, gamma) equals 1 when gamma > 1 and α (kappa, gamma) is between 0 and 1 when kappa > 1 and gamma > 1.

  5. The system can utilize the derived harmonic potential u delta to determine the optimal bias-variance trade-off by identifying u0(0), which dictates the minimax exponent alpha(kappa, gamma).

  6. A specific estimator, Bocca with gK defined by cosh (Kq delta) and hyperbolic cosine functions, is proposed to achieve the optimal rate n minus alpha(kappa, gamma), where K is chosen to balance the bias and variance terms as specified in Theorem 3.6.

  7. The lower bound established by Theorem 4.1 shows that for any estimator of a fixed linear functional βv, the minimax rate is at least n minus alpha(kappa, gamma), providing a benchmark for performance evaluation.

  8. The system can construct preliminary Gaussian experiments with covariance eigenvalues in [kappa minus 1, 1] and scale them to match the problem family P(n, p, gamma, kappa), verifying that these experiments achieve the lower bound of n minus alpha(kappa, gamma)/2.

  9. The framework allows for the construction of supported priors Q0 and Q1 whose data laws satisfy TV(P0, P1) less than or equal to 1/3 while ensuring b1 minus b0 is at least ce minus Kq(0) slash n minus alpha (kappa, p/n)/2.

  10. The system can generate the final Gaussian regression parameters using a mixing distribution Pi(V) and conditional regressions where the noise variance sigma 2 j is bounded within the interval four varj s0 over varj plus s0, satisfying 4s0 over three less than or equal to sigma 2 j less than or equal to two s0.

  11. The system can verify that the total variation distance between data laws PeC j and Pej is at most C tau (ne minus two Kh plus K squared slash n) one half plus Ce minus cn u squared, which is shown to be at most 1/4 under condition F(1).

Abstract

We study estimation of a linear functional in random-design linear models when γ=p/n in (1, infinity) under the dense regime where the design covariance is unknown, and neither the regression vector nor the precision matrix is assumed sparse. Suppose the spectrum of the design covariance belongs to [,L], and let κ=L/. We establish that the minimax mean squared error for the linear functional is of order n-α(κ,γ), where the exponent α(κ,γ) has an explicit elliptic-integral representation and α(κ,γ) < 1 when both κ> 1 and γ> 1. We propose the Bocca estimator, which combines empirically calibrated resolvents through a complex contour integral to attain the optimal rate. We also establish asymptotic normality of the Bocca estimator and develop a variance estimator for valid inference.

Sources

Related papers