dsp: An R Package for Bayesian Time Series Modeling with the Dynamic Shrinkage Process

arXiv:2611.07096 · stat.CO, stat.ME · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "dsp: An R Package for Bayesian Time Series Modeling with the Dynamic Shrinkage Process".

Jane: Here's the gist of the paper. dsp provides a unified state-space framework for modeling nonstationary time series using the Dynamic Shrinkage Process (DSP), a class of global-local shrinkage priors This package offers a common interface for model specification, posterior computation, summarization, and visualization across models that were previously developed separately in the literature Core Methodology The DSP is defined by a hierarchical form where the local scale parameter follows an autoregressive process.

Tom: The local term is defined as

ωt vt: indep ∼ N(0, exp(vt)).

Jane: The local parameter ηt follows a Z-distribution parameterized as Z(α, β, mz, σz).

Tom: The distinctive feature of DSP is its incorporation of time dependence between the local terms, ψt.

Title and authors: Tom: So we're talking about the title "dsp: An R Package for Bayesian Time Series Modeling with the Dynamic Shrinkage Process" and who came up with it. We already touched on what DSP is, but let's focus on what this specific package does.

Jane: It was written by Jason B. Cho and Daniel R. Kowal from Cornell University, so they’re the ones driving this unified framework for modeling nonstationary time series using the Dynamic Shrinkage Process or DSP >

Lu: What I see is that it consolidates a lot of prior work on DSP-based Bayesian time series models into a single, coherent programmatic framework which simplifies how researchers can use these models >

Meng: It’s an R package, and they implemented the core functionality in C++ via Rcpp for computational efficiency, which is important because we’re dealing with large datasets and we need fast posterior computation >

Lalam: This means performance-critical components are implemented in C++ via Rcpp, enabling fast posterior computation and scalability to large data sets >

Tom: So the package offers a consistent interface for specifying models, running MCMC-based inference, and summarizing and visualizing results across trend filtering, changepoint detection, and volatility modeling >

Jane: It’s really about providing that common interface so researchers can easily switch between those different applications within a single software package >

The paper's summary: Tom: Let's look at the core summary of this work. The paper states that DSP adaptively regularizes time-varying parameters to shrink noise while preserving abrupt changes and other local structure in the time series >

Jane: It points out that nonstationary time series often have periods of relative stability punctuated by abrupt changes in mean or variance, which is exactly what this modeling approach is designed to handle >

Lu: The methodology involves a hierarchical form where the local scale parameter follows an autoregressive process, and the local term is defined as omega t vt indep ∼ N(zero exp(vt)] >

Meng: That structure means the local log-scales follow an autoregressive process rather than being independent, which compounds a difficulty because the funnel recurs at each time point and the scales are series dependent >

Lalam: The distinctive feature of DSP is incorporating time dependence between those local terms, psi t, allowing the model to dynamically adjust the degree of regularization over time >

Tom: So in short, it’s this unified state-space framework that gives us global-local shrinkage priors that adaptively regularize parameters based on their local structure and temporal dependencies >

Jane: And because of this, it supports Gaussian and count-data smoothing, changepoint detection, time-varying parameter regression, and volatility estimation with posterior predictive draws available for missing-value imputation and forecasting >

The paper's improvements: Tom: The authors discuss some specific improvements they made to the existing DSP models. They mention things like adapting the regularization strategy dynamically based on whether the data is Gaussian or count data >

Lu: They suggest that adaptive regularization in high-dimensional models can be achieved by combining a single global parameter controlling overall shrinkage with local scale parameters, which reduces noise while preserving sharp local structures in the signal >

Meng: That sounds useful because it means we can potentially adapt our regularization strategy dynamically based on the characteristics of the data, whether it's Gaussian or count data >

Lalam: They also improved changepoint detection by using models like ABCO that decompose the observed series into three components: a local mean mu t, a sparse additive outlier signal zeta t, and a heteroskedastic noise term epsilon t >

Jane: That decomposition is interesting because it allows AI systems to accurately identify abrupt shifts in mean behavior while simultaneously accounting for isolated outliers and time-varying volatility >

Conclusion: Tom: So wrapping up this paper on "dsp: An R Package for Bayesian Time Series Modeling with the Dynamic Shrinkage Process". The main implication is that we now have a single, flexible, and computationally efficient framework to model nonstationary time series across several important applications >

Jane: It gives us a consistent interface for specifying models, running MCMC-based inference, and summarizing results for things like trend estimation and volatility modeling >

Lu: The authors point out that the geometry of the posterior is generally unsuitable as written for Hamiltonian Monte Carlo samplers like Stan because global-local shrinkage priors induce a funnel-shaped posterior >

Meng: That's a big caveat. It means we can't just throw this into standard gradient-based samplers, which is something I need to keep in mind when planning deployment >

Lalam: They also provided tools for posterior predictive draws at every observation time, so missing-value imputation and forecasting are available directly from the MCMC output >

Tom: So it’s a powerful tool for analyzing complex data where we expect things to change abruptly over time, but we need to be careful about which sampling methods we use for inference >

Jane: Exactly. This paper on the dsp package provides us with a unified way to handle these kinds of nonstationary challenges, and it sets the stage for how AI can better model dynamic systems that shift over time.

Jason B. Cho, Daniel R. Kowal, Haoxuan Wu, David S. Matteson, Toryn L J Schafer

Cornell University · Headlands Technologies LLC

stat.CO, stat.ME

Submitted: 2026-11-04

Updated: 2026-11-04

Comments: 37 pages, 13 figures

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: dsp provides a unified state-space framework for modeling nonstationary time series using the Dynamic Shrinkage Process (DSP)…

Key concepts

Dynamic Shrinkage Process (DSP)
DSP is a hierarchical model where the local scale parameter follows an autoregressive process. It incorporates time dependence between local terms, allowing the model to dynamically adjust regularization over time based on data characteristics.
Local Term [ωt | vt]
This term represents the local component of the time series at time t, conditional on a latent variable vt. It is defined independently from other local terms and follows a normal distribution whose variance depends on vt.
Unified Framework
The package offers a common interface for various models, including trend filtering, changepoint detection, and time-varying parameter regression. This means users can switch between different modeling tasks within the same software structure.
Posterior Predictive Draws
The package generates full posterior predictive draws at every observation time. These draws are essential tools for downstream applications like missing-value imputation and forecasting by leveraging the entire distribution of possible outcomes.

Terminology

Summary

dsp provides a unified state-space framework for modeling nonstationary time series using the Dynamic Shrinkage Process (DSP), a class of global-local shrinkage priors This package offers a common interface for model specification, posterior computation, summarization, and visualization across models that were previously developed separately in the literature

Core Methodology

The DSP is defined by a hierarchical form where the local scale parameter follows an autoregressive process. The local term is defined as [ωt vt] indep ∼ N(0, exp(vt)). p. 2

Where the paper says it

Dynamic Shrinkage Process Let {yt} T t=1 denote an observed univariate time series of length T, and let {ωt} T t=1 be a time-varying parameter of interest associated with {yt} T t=1. The DSP is a flexible family of global-local shrinkage priors defined by the hierarchical form: [ωt |vt ] indep ∼ N(0, exp(vt)), vt = m + ψt + ηt , [ηt |α, β] iid∼ Z(α, β, 0, 1), (1) where m is the global scale parameter and ψt+ηt is the local scale parameter.

The local parameter ηt follows a Z-distribution parameterized as Z(α, β, mz, σz). p. 2
Where the paper says it

, T from a heavy-tailed distribution, allowing selective deviation from strong shrinkage. Specifically, ηt follows the Z-distribution parameterized as Z(α, β, mz, σz) with the probability density function: f(z|α, β, mz, σz) = {σzB(α, β)} −1 exp{(z − mz)/σz} α [1 + exp{(z − mz)/σz}] −(α+β) z ∈ R, where B(α, β) denotes the Beta function.

The distinctive feature of DSP is its incorporation of time dependence between the local terms, ψt. p. 3
Where the paper says it

By additionally setting ψt = 0 and m ∼ Z(1/2, 1/2, 0, 1), DSP simplifies exactly to the Horseshoe prior (Carvalho et al. The distinctive feature of DSP is its incorporation of time dependence between the local terms, ψt , enabling dynamic modeling of the shrinkage parameter, hence the name Dynamic Shrinkage Process.

Model Flexibility and Applications

The package supports several model types including Gaussian and count-data smoothing. Specific applications include trend filtering for Gaussian data (Bayesian Trend Filter with the DHS prior) and changepoint detection (Adaptive Bayesian Changepoints with Outliers or ABCO model). p. 3

Where the paper says it

These model-specific dependence structures are separate from the choice of prior family, and their detailed parameterizations are given in the corresponding sections. Bayesian Trend Filter for Gaussian Data As one of many applications of DSP, Kowal et al.

It also supports time-varying parameter regression and volatility estimation, such as Adaptive Stochastic Volatility (ASV). p. 1
Where the paper says it

2 dsp: Bayesian Time Series Modeling with the Dynamic Shrinkage Process filtering for integer-valued time series (Schafer and Matteson 2024), and adaptive stochastic volatility (ASV) for time-varying volatility estimation (Cho and Matteson 2026b). dsp is an R package that provides a unified, flexible, and computationally efficient framework for adaptive modeling.

Computational Efficiency and Sampler Choice

Core functionality is implemented in C++ via Rcpp for computational efficiency. Model-specific samplers are chosen over general-purpose engines like Stan because closed-form full conditional distributions are available for the majority of parameters. p. 1

Where the paper says it

Our choice is primarily driven by the following two reasons. First, closed-form full conditional distributions are available for the majority of the parameters and latent variables (Kowal et al.

This choice is also driven by the geometry of the posterior, which is generally unsuitable for Hamiltonian Monte Carlo (HMC) samplers like Stan. p. 2
Where the paper says it

Schafer 3 (McCausland, Miller, and Pelletier 2011). Second, the geometry of the posterior of our model specification is generally unsuitable as written for Hamiltonian Monte Carlo (HMC) samplers such as Stan.

Model Specification and Output

The dsp package organizes its workflow around model specification, posterior computation, and analysis. Users specify the likelihood, model type, evolution error prior, and other model-specific options using the dsp spec function. p. 7

Where the paper says it

Package Overview The dsp package is organized around model specification, posterior computation, and posterior analysis. A user first calls dsp_spec() to specify the likelihood, model type, evolutionerror prior, and any model-specific options.

The fitted object returns a named list containing posterior MCMC draws, DIC information, and accessor functions for inspection. p. 9
Where the paper says it

Defaults to FALSE. Model Output The fitted dsp object is returned as a named list containing:

Empirical Validation

Empirical examples demonstrate the package's use in various contexts, such as estimating the time-varying mean of sunspot numbers using the BTF-DHS model and analyzing consumer price index data with ABCO. The package also provides tools for posterior predictive draws, which are used for missing-value imputation and forecasting. p. 0

Where the paper says it

DSP adaptively regularizes time-varying parameters to shrink noise while preserving abrupt changes and other local structure. The package supports Gaussian and count-data smoothing, changepoint detection, time-varying parameter regression, and volatility estimation, with posterior predictive draws available for missing-value imputation and forecasting.

Discussion on Model Selection

Guidance is provided on model specification, suggesting heuristics based on data context and visual inspection rather than relying solely on information criteria. For instance, the choice between different degrees of differencing (D=1 or D=2) for trend smoothing is guided by how the resulting posterior mean fits the known data-generating structure. p. 3

Where the paper says it

Trend filtering extends this idea by replacing the first-order difference with a higher-order difference, ωt = ∆Dµt , where D denotes the order of differencing. The choice of D determines the local structure of the estimated trend.

The package is designed with computational practicality and extensibility in mind, scaling linearly with the length of the time series for Gaussian models. p. 1
Where the paper says it

2019; Schafer and Matteson 2024; Wu et al. Furthermore, the conditional independence structure of the state equation renders the precision matrix of the latent state banded, which our sampler exploits to draw the entire state exactly from its joint full conditional with compute time scaling linearly in the sample size

Computational Details

The results were obtained using R 4.5.3 with Apple clang version 21.0.0 for C++. The latent state is drawn through a sparse Cholesky factorization implemented with Eigen and compiled from source at installation. p. 1

Where the paper says it

2019; Schafer and Matteson 2024; Wu et al. Furthermore, the conditional independence structure of the state equation renders the precision matrix of the latent state banded, which our sampler exploits to draw the entire state exactly from its joint full conditional with compute time scaling linearly in the sample size

Markov chain Monte Carlo amplifies such a difference, with the consequence that exact reproduction of the values reported here requires the platform described above. p. 0
Where the paper says it

We describe the underlying models, package design, and the main functions for fitting, summarizing, and plotting posterior results. Keywords: Bayesian Inference, Time Series Analysis, Gibbs Sampling, Markov Chain Monte Carlo (MCMC), State-Space Modeling.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive regularization in high-dimensional models can be achieved by combining a single global parameter controlling overall shrinkage with local scale parameters, which reduce noise while preserving sharp local structures in the signal. This allows AI systems to adapt their regularization strategy dynamically based on the characteristics of the data, whether it's Gaussian or count data.

  2. Unified framework for modeling nonstationary time series can be implemented by providing a common interface for model specification, posterior computation, summarization, and visualization across models that were previously developed separately in the literature. This enables researchers to easily switch between trend filtering, changepoint detection, and time-varying parameter regression within a single software package.

  3. Missing value imputation can be performed directly from MCMC output because the package returns full posterior predictive draws at every observation time, so missing-value imputation and forecasting are available directly from the MCMC output. This allows for robust handling of incomplete datasets in AI applications by leveraging the entire posterior distribution for prediction.

  4. Time-varying parameter estimation can be enhanced by using a model that incorporates temporal dependence in the shrinkage process through the additional time-varying term psi t, allowing the model to dynamically adjust the degree of regularization over time. This feature is particularly useful when dealing with economic or financial data where structural changes occur over time.

  5. Changepoint detection can be improved by using models like ABCO that decompose the observed series into three components: a local mean mu t, a sparse additive outlier signal zeta t, and a heteroskedastic noise term epsilon t. This allows AI systems to accurately identify abrupt shifts in mean behavior while simultaneously accounting for isolated outliers and time-varying volatility.

  6. Volatility estimation can be made more robust by allowing the observation error variance sigma squared epsilon t to be specified as either constant or time-varying, supporting models like Adaptive Stochastic Volatility (ASV) which can capture significant shifts and sudden changes within the volatility process. This is crucial for financial time series where noise characteristics change over time.

  7. Regression models with time-varying coefficients can be fitted using a framework where the locally adaptive shrinkage property of DSP makes it particularly well-suited for modeling such time-varying parameters. This allows AI systems to model how the strength of association between a response and its predictors evolves over time, as seen in the example of NVIDIA stock returns.

Abstract

dsp provides a unified state-space framework for modeling nonstationary time series using the Dynamic Shrinkage Process (DSP), a class of global-local shrinkage priors. DSP adaptively regularizes time-varying parameters to shrink noise while preserving abrupt changes and other local structure. The package supports Gaussian and count-data smoothing, changepoint detection, time-varying parameter regression, and volatility estimation, with posterior predictive draws available for missing-value imputation and forecasting. Nonstationary time series often contain periods of relative stability punctuated by abrupt changes in mean or variance. dsp offers a common interface for model specification, posterior computation, summarization, and visualization across models that were previously developed separately in the literature. Core functionality is implemented in C++ via Rcpp for computational efficiency. We describe the underlying models, package design, and the main functions for fitting, summarizing, and plotting posterior results.

Related papers