Estimation and inference in penalized function-on-function regression with within-curve residual dependence
summary
The gist
Estimation and inference in penalized function-on-function regression with dependent residual curves addresses the tension between estimation and pointwise inference in nonparametric regression by…
In short
The study investigates estimating and inferring parameters in function-on-function regression where residual curves within a single curve are dependent. Standard methods fail because they assume independence between grid points, leading to inaccurate confidence intervals. The authors propose using Curve-Clustered Sandwich Covariance (CL2) with leverage adjustments for better inference and recommend using NCV for shape estimation.
Key concepts
- Within-curve dependence
- The residual curves are not independent; values close to each other on the same curve are correlated. Standard models ignore this, causing standard errors to be too small and confidence intervals to be too narrow.
- REML vs. NCV for estimation
- Restricted Maximum Likelihood (REML) often undersmoothing occurs when dealing with dependent errors, leading to error estimates that are much larger than those from Curve-Blocked Neighbourhood Cross-Validation (NCV). NCV is recommended to accurately describe the underlying shape of the regression surface.
- CL2 Covariance and Leverage Adjustment
- The CL2 covariance method uses a curve-clustered sandwich covariance structure combined with a leverage adjustment for the penalized hat matrix. This technique helps correct for within-curve dependence, bringing confidence interval coverage close to the desired nominal level in simulations.
Terminology used across episodes
This episode discusses
- Estimation and inference in penalized function-on-function regression with within-curve residual dependence · Paper Radio
- Estimating Semiparametric and Nonparametric Fixed Effects Panel Data Models with mgcv
- Fast Penalized Generalized Estimating Equations for Large Longitudinal Functional Datasets
- On Neighbourhood Cross Validation
The paper
Estimation and inference in penalized function-on-function regression with within-curve residual dependence · Read on arXiv
Department of Statistics, LMU Munich · Munich Center for Machine Learning (MCML)
Penalized function-on-function regression, as implemented in the R package refund, considers all grid values as independent conditional on the linear predictor, but some dependence typically remains. We study the consequences for estimates and pointwise intervals in a simulation and a plasmode study on nine real datasets. Under dependence, restricted maximum likelihood (REML) undersmooths the coefficient surface: in our simulations' main settings, its relative error is two to four times that of curve-blocked neighborhood cross-validation (NCV). In our plasmode study, both fits have large errors, possibly through weak identification, and NCV is usually, but not always, better. Under dependence, the independence-model-based intervals for means and surfaces undercover severely. We propose and evaluate a curve-clustered sandwich covariance with a leverage adjustment for the penalized hat matrix with Satterthwaite degrees of freedom which brings coverage close to nominal in most settings. Its intervals for function-on-function effects are wide because REML's estimate is noisy. The same construction at the NCV estimate is narrow but undercovers. We therefore recommend fitting twice: NCV for the shape of the surface, REML with these intervals for inference. For estimated means, intercept functions and scalar covariate effects, one REML fit with these intervals usually suffices.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Estimation and inference in penalized function-on-function regression with within-curve residual dependence".
Jane: Here's the gist of the paper. Estimation and inference in penalized function-on-function regression with dependent residual curves addresses the tension between estimation and pointwise inference in nonparametric regression by analyzing how within-curve dependence affects point estimates and confidence intervals.
Tom: Model-based 95% intervals cover 0.63–0.82 under within-curve dependence, while a curve-clustered sandwich covariance with a leverage adjustment for the penalized hat matrix (CL2) brings coverage close to nominal in simulations.
Jane: Model Limitations and Dependence Effects The standard model treats all grid values of all curves as independent, which is incorrect because residual curves are smooth and therefore values within a curve are dependent.
Tom: This dependence leads to two consequences: first, model-based standard errors count N independent observations where the data carry information closer to G, resulting in intervals that are too narrow.
Title and authors: Tom: So we’re looking at this paper titled "Estimation and inference in penalized function-on-function regression with within-curve residual dependence," and it’s a bit technical, but the core issue is how we estimate things when the data points are actually related because they come from smooth curves.
Jane: Right, so what does that mean for us when we're doing this kind of function-on-function regression? Basically, standard methods assume everything is independent, but here the residual curves are smooth, so the values on one curve influence the values on its neighbors.
Lu: Exactly. The paper points out that this dependence messes with two things: first, model-based standard errors count too many independent observations where the data actually carries information closer to G, which makes your confidence intervals too narrow.
Meng: And second, when you use restricted maximum likelihood or REML to pick how smooth a function should be, it reads that dependence as if it were a real signal, so it undersmoothing happens and your estimates get noisier than they need to be <ref:2610.04677#pg2>.
Tom: It sounds like the standard tools are failing us here because they don't account for that smooth relationship within each curve, which is a problem we see when we try to deploy these models at scale <ref:2610.04677#pg1>.
Jane: So what’s the paper suggesting as a fix? They introduce this new approach called the Curve-Clustered Sandwich Covariance, or CL2, which includes a leverage adjustment for the penalized hat matrix <ref:2610.04677#pg3>.
Lu: They found that using this CL2 covariance with that leverage adjustment brings the coverage of ninety-five percent intervals close to what we expect in simulations <ref:2610.04677#pg1>.
Meng: But they also noted that for shape estimation, the authors recommend using a fit smoothed by curve-blocked neighbourhood cross-validation, or NCV, as the default because REML undersmoothing is problematic and NCV gives a better shape description <ref:2610.04677#pg3>.
Tom: That means we might need to shift our focus from just getting the best estimate with REML to using NCV for describing the surface shape itself <ref:2610.04677#pg3>.
Jane: And for inference about that surface, they suggest using REML combined with CL2 and Satterthwaite degrees of freedom for the degrees of freedom <ref:2610.04677#pg3>.
Lu: That combination actually gives both a good estimate and intervals that are close to nominal, especially for effects where the covariate changes along time <ref:2610.04677#pg3>.
Meng: They found that this REML plus CL2 setup covers ninety-four to ninety-five percent in simulations for both the mean and coefficient surfaces <ref:2610.04677#pg1>.
Tom: But they also showed that NCV plus CL2 is a little short on eight datasets, and for the coefficient surface on the ninth, it’s only eighty-four to ninety-one percent coverage <ref:2610.04677#pg1>.
Jane: So, when we look at the practical guidelines, they suggest using NCV as the default for shape estimation because it smooths more than REML by default <ref:2610.04677#pg3>.
Lu: For inference about the surface, REML plus CL2 with Satterthwaite degrees of freedom is what they suggest as the main tool <ref:2610.04677#pg3>.
Meng: They also mentioned an exception for binary responses and misregistered counts where they treat those intervals as approximate because they don't cover well <ref:2610.04677#pg3>.
Tom: That points to a real-world constraint—we can't expect perfect coverage everywhere, especially with counts or binary outcomes <ref:2610.04677#pg3>.
Jane: So, what does this mean for the world when we think about these kinds of models in general? It means if you’re using function-on-function regression where the underlying data has smooth dependencies, you need to be careful about how you interpret your uncertainty <ref:2610.04677#pg2>.
Lu: The paper is really focused on providing practical recommendations for users of tools like pffr, suggesting what defaults the package should have and what they should report <ref:2610.04677#pg3>.
Meng: From a deployment standpoint, having these specific covariance adjustments like CL2 implemented in the software would be crucial because it moves us closer to reliable inference when we’re running these kinds of models on real data <ref:2610.04677#pg1>.
Tom: And they did introduce something called the CR2 leverage adjustment from Bell and McCaffrey, which is computed with the non-idempotent hat matrix <ref:2610.04677#pg3>. That sounds like a specific mechanical fix to stabilize those errors.
Jane: So, we have these recommendations: use NCV for shape, REML plus CL2 with Satterthwaite degrees of freedom for inference, and be careful with binary data <ref:2610.04677#pg3>.
Lu: And they also suggest that if you’re dealing with scalar effects that vary along time, the combination of REML plus CL2 and Satterthwaite degrees of freedom gives both a good estimate and intervals close to nominal <ref:2610.04677#pg3>.
Meng: That's the functional effect on scalar covariates varying along time—that’s where they claim they get that balance for both things <ref:2610.04677#pg3>.
Tom: And I see some future directions mentioned, like looking at intermediate smoothing levels between REML and NCV, or doing global tests of beta equal to zero <ref:2610.04677#pg3>.
Jane: They also mentioned that for binary responses and misregistration counts, the intervals are treated as approximate because they just don't cover well <ref:2610.04677#pg3>.
Lu: And they did suggest using a richer basis when fitting functional covariates to protect against approximation bias, even though that choice widens those REML plus CL2 intervals <ref:2610.04677#pg3>.
Meng: It sounds like the paper is really digging into the messy reality of these models—how theory meets actual simulation results across different response types and data setups <ref:2610.04677#pg1>.
Tom: So, to wrap up this discussion on "Estimation and inference in penalized function-on-function regression with within-curve residual dependence," the paper shows that simply ignoring the smooth dependence between curves leads to intervals that are too narrow and estimates that are noisier <ref:2610.04677#pg2>.
Jane: The main practical advice is to use NCV for shape, but when you need inference, REML plus CL2 with Satterthwaite degrees of freedom is the recommended path because it balances estimation and interval coverage <ref:2610.04677#pg3>.
Lu: It’s about moving beyond the naive assumption of independence and incorporating structured residual covariance directly into our statistical framework, which is what this paper does by introducing the CL2 term <ref:2610.04677#pg3>.
Meng: For us in production, it suggests that implementing these kinds of robust covariance structures is what will make the difference when we move from simulation to real traffic and need reliable error bars <ref:2610.04677#pg1>.
Tom: It's a solid piece of work because it’s not just about showing a result, it’s about giving us the tools—like the CR2 leverage adjustment—to actually make sense of our estimates when things get messy <ref:2610.04677#pg3>.
Jane: So we’ve covered how dependence affects estimation, what NCV is good for shape, and what REML plus CL2 is better for inference in the context of this paper, "Estimation and inference in penalized function-on-function regression with within-curve residual dependence."
The paper's summary: Tom: So, this paper is really about how to get good estimates when you're doing function-on-function regression and those residual curves aren't independent, which they definitely aren't <ref:2610.04677#pg2>.
Jane: Right, so the main tension they’re talking about is between just getting a point estimate and actually having a reliable confidence interval that tells you how much uncertainty you have <ref:2610.04677#pg3>.
Lu: Exactly. Standard methods treat every data point as totally separate, but since these residual curves are smooth, they’re actually connected, which messes up the standard error calculations <ref:2610.04677#pg2>.
Meng: So what's the practical problem? One thing is that those standard errors end up being too small because they count too many independent observations where the data is actually dependent on each other <ref:2610.04677#pg2>.
Tom: And there’s another issue, which they point out, when you use methods like restricted maximum likelihood or REML to decide how smooth your function should be, it treats that dependence as if it were a real signal instead of just noise <ref:2610.04677#pg2>.
Jane: So the smoothing parameter selection gets skewed because the error measurement is wrong, and they say REML’s error for that coefficient surface can be two to four times bigger than what you get if you choose your smoothing parameters using a different method <ref:2610.04677#pg2>.
Lu: That’s the core of the estimation problem, and they suggest two main paths forward: NCV for describing the actual shape of the surface, and REML combined with this new curve-clustered sandwich covariance adjustment <ref:2610.04677#pg3>.
Tom: The CL2 adjustment is what brings those ninety-five percent intervals back close to what we actually expect in simulations <ref:2610.04677#pg1>. It’s a big deal because it makes the inference much more trustworthy <ref:2610.04677#pg3>.
Jane: But they also give us some caveats, like NCV might be a bit too narrow on eight out of nine datasets, and for the coefficient surface on the ninth dataset, it only hits eighty-four to ninety-one percent coverage <ref:2610.04677#pg1>.
Meng: That means we can’t just pick one tool; we have to be smart about which one you use depending on what you’re trying to do <ref:2610.04677#pg3>.
Tom: For shape estimation, the paper recommends NCV as the default because it smooths more than REML by default <ref:2610.04677#pg3>.
Jane: And for making actual inferences about that surface, they recommend using REML plus CL2 with Satterthwaite degrees of freedom for the degrees of freedom <ref:2610.04677#pg3>.
Lu: That specific combination is what they found gives you both a good estimate and intervals that are close to nominal, especially when the effect varies along time or a second argument <ref:2610.04677#pg3>.
Meng: So if I’m deploying this, I’d probably start with NCV for the shape but then run the REML plus CL2 setup for my final inference checks <ref:2610.04677#pg3>.
Tom: And they also flag that for binary responses or misregistered counts, you should just treat those intervals as approximate because they don't cover well <ref:2610.04677#pg3>.
Jane: So, the big picture is that when you’re doing this kind of regression on real data with smooth curves, ignoring that internal dependence leads to bad confidence intervals, and this paper gives us specific tools to fix that by adjusting the covariance structure <ref:2610.04677#pg3>.
Lu: It really pushes the idea that we need to incorporate these structured residual covariances directly into our statistical framework instead of treating everything as independent <ref:2610.04677#pg3>.
Meng: This feels like the kind of work you actually need when you’re trying to build something that runs reliably at scale, not just for a clean benchmark run <ref:2610.04677#pg1>.
Tom: And they even introduced a specific leverage adjustment, the CR2 adjustment from Bell and McCaffrey, which is computed using the non-idempotent hat matrix <ref:2610.04677#pg3>. That’s a mechanical fix to stabilize those errors <ref:2610.04677#pg3>.
Jane: So, we’ve seen that NCV is good for the shape, but REML plus CL2 with Satterthwaite degrees of freedom is the go-to for inference when you have these smooth dependencies <ref:2610.04677#pg3>.
Lu: And they suggest that if you’re dealing with scalar effects that change along time, this specific REML plus CL2 setup is what gives you a good estimate and intervals close to nominal <ref:2610.04677#pg3>.
Meng: It sounds like we need to be really careful about which tools we deploy based on the nature of the effect and the data type, that’s what this paper points out <ref:2610.04677#pg3>.
Tom: And they also suggested looking at intermediate smoothing levels between REML and NCV, or doing global tests for beta equal to zero <ref:2610.04677#pg3>. That’s the next step in figuring out the noise floor <ref:2610.04677#pg3>.
Jane: This paper shows that when you look at function-on-function regression, you can't just treat everything as independent and expect your confidence intervals to be right <ref:2610.04677#pg2>.
Lu: It’s about moving toward methods that respect the actual structure of the data, which is what this paper is pushing for with the curve-clustered sandwich covariance <ref:2610.04677#pg3>.
The paper's improvements: Tom: So, this paper is basically saying that we need to change how we handle those smooth residual curves because the standard math assumes everything is independent which isn't true <ref:2610.04677#pg2>.
Jane: Right, so they’re suggesting a couple of specific fixes to move away from that independence assumption and get better results <ref:2610.04677#pg3>.
Lu: They point out that the main improvement is using a curve-clustered sandwich covariance adjustment, called CL2, which includes a leverage adjustment for the penalized hat matrix <ref:2610.04677#pg3>.
Meng: So what does that actually do for production? It means we can get coverage close to what we expect in simulations when we run these models on real data <ref:2610.04677#pg1>.
Tom: But they also suggest using curve-blocked neighbourhood cross-validation, or NCV, as the default for describing the actual shape of a surface because REML undersmoothing is a problem <ref:2610.04677#pg3>.
Jane: That means for figuring out the general shape of what’s happening, you should use NCV instead of relying solely on REML to choose your smoothing parameter <ref:2610.04677#pg3>.
Lu: And then for actually inferring something about that surface, they suggest using REML combined with CL2 and Satterthwaite degrees of freedom for the degrees of freedom <ref:2610.04677#pg3>.
Meng: That specific combination gives you a good estimate and intervals that are close to nominal, especially when the effect varies along time or a second argument <ref:2610.04677#pg3>.
Tom: They also suggest using a smoothing parameter uncertainty term with REML plus CL2 to raise the coverage of the coefficient surface by about one point two to one point six percentage points without changing the interval score <ref:2610.04677#pg4>.
Jane: So, that’s a way to boost your confidence in those intervals even if the raw score doesn't change much, which is helpful for real-world decision making <ref:2610.04677#pg3>.
Lu: They also mentioned using a functional random effect per curve or Generalized Least Squares with FPCA residual covariance when REML undersmoothing is present because those methods are much better than REML and comparable to NCV in some comparisons <ref:2610.04677#pg5>.
Meng: That’s interesting because it means we have these alternatives if the basic REML setup just isn't enough for us at scale <ref:2610.04677#pg5>.
Tom: And they also said that for scalar effects that change along time, using REML plus CL2 and Satterthwaite degrees of freedom gives both a good estimate and intervals close to nominal <ref:2610.04677#pg3>.
Jane: And they did flag some limitations, saying that for binary responses or misregistered counts you should just treat those intervals as approximate because they don't cover well <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Lu: So the limitation is that this method isn't perfect for those specific data types, which is something we have to remember when we deploy it <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Meng: It’s good to know what the limitations are upfront so I don't promise ninety-nine percent coverage on a count model that just won't deliver that <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Tom: So, the big idea is shifting from just getting a number to building a statistical framework that respects the actual structure of the data, which is what this paper is pushing for with these covariance adjustments <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Jane: And it shows that by combining NCV for shape and REML plus CL2 for inference, you can get both a decent estimate and reliable intervals when dealing with these smooth dependencies <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Lu: It’s about making the tools more robust against the way data is naturally structured, which is something that will matter as we build more complex models <ref:two thousand six hundred ten point zero four six seven seven#pg3.
Conclusion: Tom: So we've just been talking about how function-on-function regression gets messy when the residual curves aren't independent, which is what this paper, "Estimation and inference in penalized function-on-function regression with within-curve residual dependence," is all about <ref:2610.04677#pg1>.
Jane: Right, so the big picture here is that you can't just use standard tools because they miss the internal structure of the data, which means your confidence intervals are either too narrow or too wide depending on what you’re trying to do <ref:2610.04677#pg2>.
Lu: Exactly. They show that by using things like curve-clustered sandwich covariance adjustments and NCV for shape estimation, you can get much more reliable results in simulations <ref:2610.04677#pg3>.
Meng: For production, it means we need to implement these specific covariance structures because they move us closer to reliable inference when we’re running models on real traffic <ref:2610.04677#pg1>.
Tom: And they gave us clear rules: use NCV for shape, but use REML plus CL2 with Satterthwaite degrees of freedom for inference when you want that balance <ref:2610.04677#pg3>.
Jane: So, the implication is that we need to be much more careful about which statistical tools we choose based on the data type and the specific goal <ref:2610.04677#pg3>.
Lu: This paper pushes us to incorporate these structured residual covariances directly into our framework instead of treating everything as independent, which is something that will matter as we build more complex models <ref:2610.04677#pg3>.
Meng: It’s good to know the limitations too, like when it doesn't cover well for binary responses or misregistered counts <ref:2610.04677#pg3>, so I don't promise perfect coverage on every single type of data <ref:2610.04677#pg3>.
Tom: We’ve seen the mechanics behind it, the CR2 leverage adjustment and the CL2 covariance definition, which is what makes these methods work better than basic REML <ref:2610.04677#pg3>.
Jane: So we’ve covered how dependence affects estimation, what NCV is good for shape, and what REML plus CL2 is better for inference in the context of this paper <ref:2610.04677#pg3>.
Lu: It really shows that when you look at function-on-function regression, you can't just treat everything as independent and expect your confidence intervals to be right <ref:2610.04677#pg2>.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates