Inherited Local Shrinkage for Adaptive Functional Form Discovery

arXiv:2611.06847 · stat.ME · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Inherited Local Shrinkage for Adaptive Functional Form Discovery".

Jane: Here's the gist of the paper. Statistical models frequently employ nonlinear effects, interactions, and multiresolution basis expansions to represent complex relationships between predictors and responses This paper introduces Inherited Local Shrinkage (ILS), a Bayesian global-local shrinkage framework for adaptive functional form discovery that incorporates strong heredity among hierarchically related predictors ILS propagates shrinkage through directed acyclic graphs (DAGs) encoding relationships among predictors, allowing increasingly complex terms to contribute only when supported by their simpler ancestors Simulation studies demonstrate that ILS achieves competitive predictive performance relative to existing heredity-constrained methods while recovering underlying functional structure Framework Introduction The proposed method propagates shrinkage through a directed acyclic graph (DAG) encoding hierarchical relationships among predictors, ensuring that increasingly complex terms receive support only when simpler parent structure is retained By embedding strong heredity directly into the local shrinkage mechanism, ILS adaptively suppresses unsupported high-order terms while preserving interpretable and well-formulated models The framework is intentionally general and applies beyond polynomial regression, accommodating any regression basis admitting a DAG representation Heredity Constraints in Polynomial Models

Paper summary: Tom: So we've talked about Inherited Local Shrinkage for Adaptive Functional Form Discovery, and the authors are Jacob C. Schultz, Daniel Taylor-Rodriguez, Fariborz Maseeh from Portland State University. They introduced this framework to handle adaptive functional form discovery in structured regression models by propagating shrinkage through DAGs based on predictor hierarchy.

Jane: It’s about giving you a common prior framework so you can control model complexity adaptively, ensuring that terms only contribute when supported by their simpler ancestors, and they showed it works for polynomial regression and wavelet bases.

Lu: The implication is that this method provides a consistent way to manage the complexity of models with nonlinear effects or interactions without having to manually tune every single term.

Meng: For deployment, it means you have a mechanism that should help keep your models stable and manageable at scale by automatically pruning unnecessary complexity based on the underlying structure of your data predictors.

Lalam: The authors showed simulation studies where ILS achieved competitive predictive performance compared to existing methods that are already constrained by heredity, and they also recovered the underlying functional structure.

Tom: So what does this mean for the world? It means for anyone trying to build complex AI models, especially in scientific applications where relationships are intricate, there’s a way to control model complexity adaptively using structural information rather than just brute-force tuning.

Jane: It’s about moving from guesswork on model structure to a framework that uses the known hierarchical relationships between predictors to guide how the model should grow in terms of functional form.

Conclusion: Tom: So we're wrapping up on Inherited Local Shrinkage for Adaptive Functional Form Discovery by Schultz, Taylor-Rodriguez, and Maseeh from Portland State University. This framework basically uses a directed acyclic graph to tell a model when it should stop adding complicated terms.

Jane: It takes these relationships between your predictors and uses them to decide which parts of the functional form are important. So, instead of guessing how many terms you need, the method shrinks the complex stuff unless the simpler structure actually supports it.

Lu: The core idea is propagating shrinkage through a DAG that shows how predictors are related hierarchically. It means if a simple relationship holds, you get support for all the more complicated terms built on top of it.

Meng: From my side, I'm interested in how this affects deployment. If the model structure is automatically pruned based on the data relationships rather than manual tuning, that could simplify the pipeline and reduce computational load when we move to production.

Lalam: I see a lot of hope here because it seems to find a good balance. The authors showed that ILS gets competitive performance against methods that already have some kind of structure constraint built in.

Tom: Competitive is good, but how does it compare when the true underlying function is really messy? Does this framework handle those nonlinear effects well, or does it still rely on the predictors being somewhat neatly organized?

Jane: The paper suggests it's quite flexible because it doesn't just stick to polynomial regression. It works for any basis that can be represented by a DAG, which is pretty broad for different types of data science problems.

Lalam: Exactly. And they recovered some of the actual functional structure in their simulation studies, which is a big deal because it means the model isn't just making predictions; it’s learning something about the underlying process.

Tom: So we've talked about how this framework works and what they achieved on paper. It really boils down to using data structure to manage complexity adaptively. What does this actually mean for a scientist or a data engineer trying to build something complex?

Jane: It means you get a way to control model complexity that adapts based on the actual predictors in your system, rather than guessing the right number of interactions upfront.

Tom: Right. It moves you away from just tuning parameters and toward using known relationships to guide the model's growth. Next up, we’re going to look at how this applies specifically to polynomial models and what those constraints actually look like in practice.

Jacob C. Schultz, Daniel Taylor-Rodriguez

Portland State University

stat.ME

Submitted: 2026-11-04

Updated: 2026-11-04

Comments: 34 pages, 9 figures, 3 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: Statistical models frequently employ nonlinear effects, interactions, and multiresolution basis expansions to represent complex relationships between predictors and responses This paper introduces…

Key concepts

Inherited Local Shrinkage (ILS)
A Bayesian framework that shrinks high-order terms in a regression model based on the support of their simpler ancestor terms. It uses a directed acyclic graph to guide this shrinkage, ensuring that more complex parts of the model are only included if the basic structure is valid.
Directed Acyclic Graph (DAG)
A diagram used to represent the hierarchical relationships among different predictors in a statistical model. In ILS, this DAG encodes which predictors influence others, allowing the method to propagate shrinkage based on these known dependencies.
Heredity Constraints
The principle that suggests complex relationships should only be included if their simpler constituent parts are also supported by the data. ILS embeds this constraint directly into its local shrinkage mechanism to adaptively suppress unsupported high-order terms while preserving model interpretability.

Terminology

Summary

Statistical models frequently employ nonlinear effects, interactions, and multiresolution basis expansions to represent complex relationships between predictors and responses This paper introduces Inherited Local Shrinkage (ILS), a Bayesian global-local shrinkage framework for adaptive functional form discovery that incorporates strong heredity among hierarchically related predictors ILS propagates shrinkage through directed acyclic graphs (DAGs) encoding relationships among predictors, allowing increasingly complex terms to contribute only when supported by their simpler ancestors Simulation studies demonstrate that ILS achieves competitive predictive performance relative to existing heredity-constrained methods while recovering underlying functional structure

Framework Introduction

The proposed method propagates shrinkage through a directed acyclic graph (DAG) encoding hierarchical relationships among predictors, ensuring that increasingly complex terms receive support only when simpler parent structure is retained By embedding strong heredity directly into the local shrinkage mechanism, ILS adaptively suppresses unsupported high-order terms while preserving interpretable and well-formulated models The framework is intentionally general and applies beyond polynomial regression, accommodating any regression basis admitting a DAG representation

Heredity Constraints in Polynomial Models

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive Functional Form Discovery using Inherited Local Shrinkage (ILS) allows AI systems to adaptively regulate model complexity across multiple resolutions by identifying where finer-scale detail is supported by the data. This enables models to determine the level of nonlinear, interaction, or multiscale complexity warranted by the data while preserving well-formulated hierarchical structure.

  2. Enhanced Hierarchical Structure Preservation in Model Selection. ILS ensures that increasingly complex terms receive support only when simpler parent structure is retained, which helps avoid the issue where a model containing an interaction term without its associated main effects can produce coefficients whose interpretations depend on arbitrary coding transformations of the predictors.

  3. Improved Predictive Performance through Adaptive Resolution Selection in Spatial and Temporal Data. The framework enables AI to use a metric termed Expected Adaptive Resolution to identify the finest resolution that contributes meaningfully to the posterior surface at each location, allowing for adaptive allocation of model complexity according to local structure in the response surface.

Related papers