Principal component error in high-dimensional factor models

summary

The gist

In this paper, principal component error in high-dimensional factor models analyzes how well true principal directions are estimated by leading eigenvectors of a sample covariance matrix in the…

In short

This study analyzes errors when estimating principal directions in high-dimensional factor models using sample covariance matrices. It decomposes the error into out-of-subspace and in-subspace components, providing data-driven bounds and limits. The findings show that the out-of-subspace error dominates, guiding practitioners on sample size requirements for accurate PCA results.

Key concepts

Out-of-Subspace Error (OE)
This measures how far a principal component estimate is from the true direction of the underlying factor. It is estimable from data and provides a lower bound for the total error, helping researchers gauge accuracy.
In-Subspace Rotation Error
This error reflects how much the fixed sample covariance matrix differs from its true population counterpart. Because it cannot be estimated solely from observations, users must provide external estimates of factors (GB and Σf) to determine its distribution.
HL Asymptotic Regime
This refers to the high-dimensional, low-sample-size limit common in finance and genomics. The paper shows that results derived under this regime offer better approximations for finite sample errors when the number of observations (n) is increased.
Data-Driven Floor
A computable lower bound for total error derived from sample eigenvalues. This floor is estimated using the average bulk eigenvalue, which serves as a reliable measure of the minimum expected error in principal component estimation.

Terminology used across episodes

This episode discusses

The paper

Principal component error in high-dimensional factor models · Read on arXiv

Alex Bernstein, Lisa R. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin, Dayi Yao

In a statistical factor model, principal components (or eigenvectors) of a sample covariance matrix serve as estimates of principal directions, the true drivers of co-movement of a collection of observed variables. We write the often substantial error in these estimates as a sum of two interpretable terms, which we show have almost sure asymptotic limits as the number of variables grows with sample size bounded. This scenario is commonplace in financial economics, genomics, machine learning and signal processing. Out-of-subspace error measures the distance from an estimate to the subspace spanned by population factor exposures. It can be expressed in terms of data, providing an estimable floor for asymptotic error. In-subspace error is driven by a rotation error within that subspace, which arises from the fixed sample size of the latent factor returns and cannot be estimated from data alone. We illustrate our error analysis with a three-factor simulation of the US public equity market, showing the dependence of the magnitude of the error and its components on dimension and sample size. In that simulation, out-of-subspace error dominates. Researchers who rely on principal component analysis to estimate factor models can use our results to quantify errors in model-based predictions and attributions.

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: We're looking at the paper: "Principal component error in high-dimensional factor models".

Emmy: Here's the gist of the paper. In this paper, principal component error in high-dimensional factor models analyzes how well true principal directions are estimated by leading eigenvectors of a sample covariance matrix in the high-dimension low-sample-size asymptotic regime, which is common in finance and genomics.

Ada: Error Decomposition The error in the jth estimated principal direction is decomposed into two interpretable terms: the out-of-subspace error, which measures the distance of a principal component to the subspace spanned by the true principal directions, and the in-subspace error, which reflects the deviation of a fixed sample covariance matrix from its population counterpart.

Emmy: Theorem 1 asserts that there is an almost sure large-p asymptotic limit for this error.

Ada: Estimable and Non-Estimable Errors The out-of-subspace error, denoted as OE, is estimable in the sense that it can be estimated from n observations of p variables, and its almost sure limits in p of the estimate and the error agree (Theorem 2).

Title and authors: Ada: So we’re looking at this paper titled "Principal component error in high-dimensional factor models." It’s about how accurately we can estimate the true principal directions when we have way more variables than observations, which is super common in finance and genomics.

Emmy: And the authors, Bernstein, Goldberg, Gunther, Kercheval, Lan, and Yao. They are really diving into this error analysis to figure out what’s happening in that high-dimensional low-sample-size asymptotic regime where p grows much faster than n.

Kolya: It sounds like they are trying to quantify the error in those principal component estimates by breaking it down into two parts, which is a key first step.

Ada: Exactly. They break the error into two pieces: the out-of-subspace error and the in-subspace rotation error. The paper argues that both of these errors have almost sure asymptotic limits as p goes to infinity with n fixed > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model, principal components (or eigenvectors) of a sample covariance matrix serve as estimates of principal directions <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Emmy: The out-of-subspace error measures how far the estimate is from the true subspace spanned by the population factors. That’s one thing we can measure directly with our data.

Kolya: And then there's the in-subspace error, which comes from having a fixed sample size for those latent factor returns, and that part they say you just can't estimate it from the data alone > page two of that work reads: "Our first result, Theorem one asserts that there is an almost sure large-p asymptotic limit for this error <ref:2609.20550#pg2,Our first result, Theorem 1, asserts that there is an almost sure>. Denote by bj the jth strongest principal direction...".

Ada: That’s where it gets interesting. Theorem two shows that the out-of-subspace error can be approximated using the eigenvalues of the sample covariance matrix <ref:2609.20550#pg2,that the out-of-subspace>. This gives us a data-driven lower bound for that total error > page two of that work reads: "Theorem two shows that OEj can be approximated in terms of eigenvalues of a sample covariance matrix <ref:2609.20550#pg2,Theorem 2 shows that OEj can be approximated in terms of eigenvalues>."

Emmy: That sounds like they are giving us something concrete we can compute, even if the total error is bigger. It’s like finding the minimum amount of noise we should expect just based on the sample data.

Title and authors: Kolya: So, for someone looking at this, it helps answer how many observations n you actually need to get accurate estimates for a specific application, because they show how much that error scales with p and n > page two of that work reads: "In contrast to a spiked model, a factor model is specified in terms of a data generating process that explicitly separates" <ref:2609.20550#pg2>.

Ada: And then there’s the rotation error. They state in Theorem three that this rotation error can take on every value between zero and one depending on how you change the population factor covariance matrix > page two of that work reads: "In contrast to a spiked model, a factor model is specified in terms of a data generating process that explicitly separates" <ref:2609.20550#pg2>.

Emmy: So, for anyone who needs to know the rotation error distribution, they're telling us upfront that you’ll need an outside estimate for the factor covariance matrix and the true number of factors > page three of that work reads: "two Related literature and new contributions two point one Factor and spiked models Statistical factor models and their close cousins, spiked models (or spiked covariance models) make high dimensional data tractable <ref:2609.20550#pg2>."

Kolya: And looking at the simulations, they used one thousand independent nested p-trajectories to show that even with just a year's worth of daily data, like two hundred fifty observations, the errors in principal directions were still large enough to be a problem for people using PCA > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Ada: They found that the out-of-subspace error was the main driver of the total error in those simulations, and interestingly, the data-driven estimate of that error was often smaller than the actual error itself > page one of that work reads: "In a statistical factor model, principal components (or eigenvectors) of a sample covariance matrix serve as estimates <ref:2609.20550#pg2,that the out-of-subspace>...".

Emmy: And they showed how things change if you increase the number of observations n from twenty to two hundred fifty when p is fixed at three thousand variables > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Title and authors: Kolya: It seems that increasing n significantly reduces the average errors for all factors when p is held steady, but the out-of-subspace component still accounts for eighty to ninety-three percent of that total error > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Ada: They also showed that the difference between the errors calculated for finite p and the asymptotic limits gets smaller steadily as n increases, suggesting that the HL asymptotic regime is a better way to approximate these values when you have those numbers > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Emmy: So what this means for the structure of these models is that we can use the eigenvalues of the sample covariance matrix to get a computable floor for our error, which is really useful when you're dealing with high dimensions > page two of that work reads: "Our first result, Theorem one asserts that there is an almost sure large-p asymptotic limit for this error <ref:2609.20550#pg2,Our first result, Theorem 1, asserts that there is an almost sure>."

Kolya: And the in-subspace rotation error part remains tricky because it cannot be estimated from the data alone, which means we definitely need external information to model that > page two of that work reads: "In contrast to a spiked model, a factor model is specified in terms of a data generating process that explicitly separates" <ref:2609.20550#pg2>.

Ada: So for practical use, this paper gives us a tool—the ratio between the average bulk eigenvalue and the jth eigenvalue of W p—as an estimator for the out-of-subspace error which acts as that lower bound for total error > page two of that work reads: "Theorem two shows that OEj can be approximated in terms of eigenvalues of a sample covariance matrix <ref:2609.20550#pg2,Theorem 2 shows that OEj can be approximated in terms of eigenvalues>."

Title and authors: Emmy: That ratio is strongly consistent and fully observable, which means it's a reliable way to estimate the floor for the total error when you're dealing with those high-dimensional settings > page two of that work reads: "Theorem two shows that OEj can be approximated in terms of eigenvalues of a sample covariance matrix <ref:2609.20550#pg2,Theorem 2 shows that OEj can be approximated in terms of eigenvalues>."

Kolya: If you want to do path-conditional analysis, they even mention Proposition two which allows you to compute an estimated distribution for the in-subspace angular error if you know the true factor covariance matrix and number of factors > page three of that work reads: "two Related literature and new contributions two point one Factor and spiked models Statistical factor models and their close cousins, spiked models (or spiked covariance models) make high dimensional data tractable <ref:2609.20550#pg2>."

Ada: So to wrap up, this paper on the Principal component error in high-dimensional factor models shows us how to use asymptotic limits to get a lower bound for the total error by using observable statistics from the sample covariance matrix > page two of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2,lower bound for the total error>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Emmy: It really emphasizes that the out-of-subspace error is the dominant part of the error when you have high dimensions and low sample sizes > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2,that the out-of-subspace>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Kolya: To finish up my thoughts on the practical side, the paper gives us guidance for practitioners to gauge the magnitude of that finite sample error in economic decisions and scientific conclusions > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Title and authors: Ada: That’s the gist of it. It highlights the utility of high-dimensional low-sample-size asymptotics by showing that we can approximate the out-of-subspace error using eigenvalues and that this serves as a data-driven lower bound for the total error > page two of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2,serves as a data-driven lower bound for the total error>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

Emmy: It’s about using the structure of the sample covariance matrix to get a computable floor for the error while acknowledging that some parts of it, like the in-subspace rotation error, still require us to bring in external knowledge > page two of that work reads: "Our first result, Theorem one asserts that there is an almost sure large-p asymptotic limit for this error <ref:2609.20550#pg2,Our first result, Theorem 1, asserts that there is an almost sure>."

Kolya: We need to remember that the paper notes that the in-subspace rotation error cannot be estimated from data alone because it depends on the population factor covariance matrix > page two of that work reads: "In contrast to a spiked model, a factor model is specified in terms of a data generating process that explicitly separates" <ref:2609.20550#pg2,cannot be estimated from data alone>.

Ada: Exactly. So, if you’re trying to build more robust latent factor models, this tells you exactly when you need an exogenous estimate for the number of factors and the true factor covariance matrix > page three of that work reads: "two Related literature and new contributions two point one Factor and spiked models Statistical factor models and their close cousins, spiked models (or spiked covariance models) make high dimensional data tractable <ref:2609.20550#pg2>."

Emmy: And it gives us a way to improve model selection by using the ratio between the average bulk eigenvalue and the jth eigenvalue of W p as a strongly consistent, fully observable estimator for that out-of-subspace error > page two of that work reads: "Theorem two shows that OEj can be approximated in terms of eigenvalues of a sample covariance matrix <ref:2609.20550#pg2,Theorem 2 shows that OEj can be approximated in terms of eigenvalues>."

Kolya: So, the paper isn't just about finding an error bound; it’s providing a concrete way to measure how much error we have based on the structure of our data > page one of that work reads: "Principal component error in high-dimensional factor models Alex Bernstein, Lisa R <ref:2609.20550#pg2>. Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao First version: July eleven two thousand twenty-six This version: September fourteen two thousand twenty-six Abstract In a statistical factor model <ref:2609.20550#pg1,Goldberg, Nicholas Gunther, Alec Kercheval, Tian Lan, Yian Lin and Dayi Yao>...".

The paper's summary: Ada: So, to recap this paper, they’re looking at how well we can actually estimate those principal directions when you have way more variables than observations in a factor model.

Emmy: Right, they’re breaking that estimation error into two parts—the part where the estimate is off because it doesn't match the true underlying structure, and another part related to how noisy our sample data is compared to the real population data.

Ada: Exactly. And the big point they make is that we can actually measure those errors, even if they don't always give us a perfect answer. They show there’s this almost sure limit for the error as you have more and more variables, which helps us set expectations for what our PCA estimates are going to look like in the long run.

Emmy: That’s the structure behind it, really. They highlight that one piece of error, called the out-of-subspace error, is something we can estimate using just the eigenvalues from our sample covariance matrix.

Kolya: And that’s where I get interested—it means we don't have to guess how big the noise is; we can calculate a lower bound for it. They use this as a kind of floor for the total error, which is useful when you’re trying to figure out if your estimates are good enough for something real, like making an investment decision.

Ada: Exactly. So, they give us a concrete statistic—a ratio involving the average bulk eigenvalue and the jth eigenvalue of that sample covariance matrix—that acts as a strongly consistent estimator for that out-of-subspace error.

Emmy: It’s like giving you a reliable tool to measure the size of the gap between your data's structure and what you *think* is there. That ratio is consistently observable, which means it's not some random fluke; it’s tied directly to the data we have.

Kolya: But they also warn us about that other error part—the in-subspace rotation error. They say you can’t estimate that from just the data because it depends on the true population factor covariance matrix, which is something you don't know.

Ada: Right, so while we can get a solid number for the out-of-subspace error using sample statistics, to know about that rotation error—that angle of deviation—we still need some outside knowledge about the true model.

Emmy: That’s the trade-off they highlight. You get a computable lower bound for one part of the problem, but you still need to bring in external information when you want to understand the other part of it.

Kolya: It shows that for building these factor models, you need to decide what kind of knowledge is available—if you can only rely on data, or if you can incorporate some assumed structure about the factors themselves.

Ada: So they really underscore how important it is to know which error component dominates your total error when dealing with high dimensions and small samples. That’s a big deal for anyone trying to use PCA in finance or genomics where those settings are common.

Emmy: And that’s the hook for what’s next—the simulation results show that even with only a year of daily data, these errors can be quite large, which is why this kind of error quantification is so important right now.

The paper's improvements: Ada: So, we’re looking at how to actually make this error analysis better, what improvements they suggest for the paper we just heard about principal component error in high-dimensional factor models.

Emmy: They’re trying to fix the limitations we discussed, especially around those non-estimable parts and making the computable bounds even more robust.

Ada: First, they want to improve that floor for total error using a different statistic. They suggest using the ratio between the average bulk eigenvalue and the jth eigenvalue of that sample covariance matrix as a stronger way to estimate the out-of-subspace error.

Kolya: That sounds like it’s making that estimate more reliable, moving it from just an approximation toward something that is strongly consistent and fully observable, which is what we need for practical work.

Emmy: Exactly. It turns that eigenvalue ratio into a better way to track the error than just looking at a single component in isolation. It’s about creating a more robust estimator for that lower bound.

Ada: And they also want to address the rotation error issue by proposing something for path-conditional analysis using Proposition two which lets you estimate the distribution of that angular error if you already know the true number of factors and the true factor covariance matrix.

Kolya: That’s useful because it moves us closer to being able to do more detailed path-by-path analysis without needing those external inputs we talked about earlier. It's making the model itself more complete, even if it still needs some assumptions on the population structure.

Emmy: So they’re suggesting a way to bridge the gap between what we can measure directly from the sample and what we need to know about the true underlying world generating those factors.

Ada: It’s a multi-pronged approach—improving how we estimate that measurable part, and offering a pathway for analyzing that unmeasurable part when you have more information than before.

Kolya: The implication is that if you’re building these models, you can start to gauge the error magnitude much more accurately based on what your data actually gives you. That’s a big step toward making these high-dimensional factor models useful tools instead of just theoretical constructs.

Conclusion: Ada: So we’re wrapping up this look at "Principal component error in high-dimensional factor models." Essentially, they show how to use asymptotic limits to get a lower bound on the total error by looking at the sample covariance matrix statistics.

Emmy: Right, it’s about getting a computable floor for the noise when you have too many variables compared to your data points. It really connects the theoretical structure of high-dimensional spaces with what we can actually calculate from real data.

Kolya: And from a probabilistic side, that floor they are giving us—that lower bound—is governed by a certain tail behavior, and they’re showing that those constants stay reasonable even when you apply this to real applications.

Ada: It changes how we think about risk in these models because it gives practitioners a concrete way to gauge the size of the error before they even start their analysis.

Emmy: It moves us from just saying "the error is big" to actually telling someone, "based on this measurement record, you should expect at least this much noise." That’s a significant shift in how we communicate uncertainty.

Kolya: I just think it’s important that they clearly separate the part we can estimate from the part that requires external knowledge about the true factor structure. That distinction is key for model building.

Ada: True, and that distinction guides us toward needing those exogenous estimates of things like the population factor covariance matrix if we want to model that rotation error properly.

Emmy: So it’s a practical tool for model selection and validation at the same time—you can use what you have, but you know exactly what information you’re missing.

Kolya: It shows that even in these highly complex settings, there are still computable quantities that survive the asymptotic limits, which gives us some hope for applying this to real-world problems.

Ada: So to sum it up, this paper on "Principal component error in high-dimensional factor models" provides a way to use observable statistics from your sample covariance matrix as a data-driven lower bound for the total estimation error.

Emmy: It’s about using the structure of those eigenvalues to get that floor while acknowledging that some parts of it still require you to bring in external knowledge.

Kolya: The main implication is guidance for practitioners on how much finite sample error you should actually expect when making economic or scientific conclusions.

Ada: And it really emphasizes the utility of high-dimensional low-sample-size asymptotics by showing we can approximate that out-of-subspace error using eigenvalues.

Emmy: It’s about using the sample covariance matrix structure to get a computable floor for the error while acknowledging that some parts of it still require us to bring in external knowledge.

Kolya: And if you want to do path-conditional analysis, they even mention a way to estimate the distribution for that angular error if you know what's going on with those true factors.

Ada: So this paper on "Principal component error in high-dimensional factor models" shows us how to use asymptotic limits to get a lower bound for the total error by using observable statistics from the sample covariance matrix.

Emmy: It really emphasizes that the out-of-subspace error is the dominant part of the error when you have high dimensions and low sample sizes.

Kolya: To finish up, this paper gives us guidance for practitioners to gauge the magnitude of that finite sample error in economic decisions and scientific conclusions.

More episodes

← Home