Bootstrapping cascaded random matrix models: correlations in permutations of matrix products

arXiv:2405.02541 · math-ph, math.MP, physics.comp-ph, physics.data-an, physics.optics · Submitted 2024-05-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematical Physics. The day in mathematical physics: rigorous models of physical systems.

Kai: I'm Kai.

Mira: And I'm Mira, here with Lev.

Kai: We're looking at the paper: "Bootstrapping cascaded random matrix models".

Mira: Here's the gist of the paper. Bootstrapping cascaded random matrix models: correlations in permutations of matrix products is proposed to speed up statistical studies of scattering in thick random media by using a dual pool based bootstrapping approach Computational modelling of wave scattering in random disordered media is a difficult problem that has been researched in earnest in recent decades Random matrix approaches, in particular, typically sacrifice system-specific details in favour of significantly reduced computation times, allowing statistical properties to be numerically investigated Cascaded random matrix models can describe the linear scattering properties of a system through the correctly-ordered product of individual transfer matrices Cascaded models can, for example, describe universal conductance fluctuations and naturally introduce a notion of length to the system such that variations in total transmission and reflection with system size can be modelled Although allowing a more physically accurate description of a large variety of scattering systems, cascaded random matrix techniques can suffer from reduced speed since calculation of the transfer matrix for a single realisation of a thick medium can necessitate generation of a large number of thin slab matrices Moreover, statistical convergence is slower for large thicknesses due to the greater variances typically involved thereby requiring more realisations to be simulated and hence entailing longer computation times.

Kai: Dual Pool Bootstrapping Methodology The dual pool approach can be adopted whereby the pool P1 of N1 thin-slab matrices is again pre-generated, however, additionally a second pool P2 of N2 transfer matrices T∆ for sections of intermediate thickness ∆L is calculated Each matrix in P2 is found by cascading n∆ = ∆L/δL thin-slab matrices which are themselves drawn from P1 The dual pool approach then allows the number of matrix products to be reduced to N2n∆ + Nr(N∆ − 1) where N∆ = L/∆L By way of an example of potential gains, we note that in our earlier work

34: use of the dual pool approach allowed the number of matrix generations and products to be reduced by approximately three and two orders of magnitude respectively.

Mira: Estimator Bias Analysis When sampling matrices from the (finite) pool P, however, there is a non-zero probability that the same matrix is drawn multiple times when sampling is performed with replacement, i.e., there may be repeated indices in α(l) Such matrix repetitions, in general, will destroy the unbiased nature of our estimator Critically, the bias originates from the finite size of the pool from which we sample X(j) and would not arise for on-the-fly calculations and is thus an artifact of the bootstrapped approach A notable exception is when the X matrices belong to a zero mean Gaussian ensemble and each matrix product Zα is formed from the product of an odd number of matrices (N is odd).

Kai: Estimator Variance and Correlations Additional fluctuations in Fˆ, however, can arise when different samples of Zα are, to some degree, correlated To assess the strength of these additional fluctuations, we again consider the simple case for which f(Z) = Z The total covariance C(K, M, N) hence characterises the degree to which Z(1,…,N) and Zα are correlated for a given permutation α Observing further that∥var

Z: ∥1 = C(N, M, N), the relative magnitude of each contribution in the second term of Eq.

Paper summary: Kai: So we're talking about this paper, "Bootstrapping cascaded random matrix models: correlations in permutations of matrix products." The main thrust is proposing a dual pool based bootstrapping approach to speed up statistical studies of scattering in thick random media.

Mira: They are tackling the problem where cascading multiple random matrices for thin media descriptions gets computationally heavy, and statistical convergence slows down for thick media because the variances are so large.

Kai: The thesis is that by pre-generating pools of matrices and using a dual pool to construct intermediate transfer matrix realizations, you can drastically reduce both the number of matrix generations and the total number of products required for a single realization.

Mira: They are also investigating how this pool-based bootstrapping affects the statistical estimates themselves, specifically looking at estimator bias and additional fluctuations that arise from correlated samples.

Lev: The paper sets out to quantify these computational gains while simultaneously assessing the extent to which using this pool-based approach can degrade the statistical estimates of mean scattering properties.

Kai: It matters because it shows that even though bootstrapping introduces statistical noise, the consequences of spurious correlations and increased estimator covariance tend to decrease as you increase the dimension of random matrices and the number of matrix products.

Mira: So, for researchers studying optical scattering or electron transport in thick disordered media, this gives them a practical way to get more statistically reliable results without incurring prohibitive computation times.

Conclusion: Kai: The authors, Byrnes, Greaves, and Foreman at Nanyang Technological University, put forward this work on bootstrapping cascaded random matrix models. They show how to use pre-calculated pools to speed up the calculation of these scattering properties.

Mira: It’s essentially a way to trade off computation time for statistical accuracy in complex systems by reducing the number of matrix operations needed, even when you have inherent statistical challenges with sampling from finite pools.

Lev: For anyone working on quantum hardware or error correction, this points toward structuring the simulation pipeline in a way that leverages pre-computation to handle the complexity of thick media models more efficiently.

Kai: So, it's about making these large-scale scattering studies more accessible by showing that you can achieve faster results while keeping the statistical estimates reasonably sound.

School of Electrical and Electronic Engineering, Nanyang Technological University · School of Physical and Mathematical Sciences, Nanyang Technological University · Institute for Digital Molecular Analytics and Science

math-ph, math.MP, physics.comp-ph, physics.data-an, physics.optics

Submitted: 2024-05-04

Updated: 2024-05-04

Comments: 1 figure, 3 tables, 13 pages, submitted to journal 11 Sep 23. Supplementary movies available upon request

Journal ref: Phys. Rev. E 110, 015308 (2024)

DOI: 10.1103/PhysRevE.110.015308

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: Bootstrapping cascaded random matrix models: correlations in permutations of matrix products is proposed to speed up statistical studies of scattering in thick random media by using a dual pool based…

Key concepts

Cascaded Random Matrix Models
These models describe wave scattering by correctly ordering individual transfer matrices. They are useful for studying universal conductance fluctuations and naturally incorporate system length, allowing researchers to model scattering properties across various system sizes.
Dual Pool Bootstrapping
This technique reduces the number of matrix products needed for simulations. It involves pre-generating a pool of thin-slab matrices and then calculating intermediate transfer matrices by cascading these initial matrices, significantly cutting down computational time compared to standard methods.
Matrix Reuse Bias
When sampling from a finite pool, there is a risk that the same matrix is drawn multiple times. This repetition can introduce bias into the estimator. However, this bias is an artifact of bootstrapping and tends to decrease in severity as the size of the random matrices increases.
Permuted Matrix Correlations
The frequency function shows that correlations between different matrix products are non-zero and depend on how many matrices are involved. Importantly, the relative frequency of higher correlation cases decreases as the number of matrices in a product gets larger.

Terminology

Summary

Bootstrapping cascaded random matrix models: correlations in permutations of matrix products is proposed to speed up statistical studies of scattering in thick random media by using a dual pool based bootstrapping approach

Introduction to the Problem

Computational modelling of wave scattering in random disordered media is a difficult problem that has been researched in earnest in recent decades Random matrix approaches, in particular, typically sacrifice system-specific details in favour of significantly reduced computation times, allowing statistical properties to be numerically investigated Cascaded random matrix models can describe the linear scattering properties of a system through the correctly-ordered product of individual transfer matrices Cascaded models can, for example, describe universal conductance fluctuations and naturally introduce a notion of length to the system such that variations in total transmission and reflection with system size can be modelled Although allowing a more physically accurate description of a large variety of scattering systems, cascaded random matrix techniques can suffer from reduced speed since calculation of the transfer matrix for a single realisation of a thick medium can necessitate generation of a large number of thin slab matrices Moreover, statistical convergence is slower for large thicknesses due to the greater variances typically involved thereby requiring more realisations to be simulated and hence entailing longer computation times.

Dual Pool Bootstrapping Methodology

The dual pool approach can be adopted whereby the pool P1 of N1 thin-slab matrices is again pre-generated, however, additionally a second pool P2 of N2 transfer matrices T∆ for sections of intermediate thickness ∆L is calculated Each matrix in P2 is found by cascading n∆ = ∆L/δL thin-slab matrices which are themselves drawn from P1 The dual pool approach then allows the number of matrix products to be reduced to N2n∆ + Nr(N∆ − 1) where N∆ = L/∆L By way of an example of potential gains, we note that in our earlier work [34] use of the dual pool approach allowed the number of matrix generations and products to be reduced by approximately three and two orders of magnitude respectively.

Estimator Bias Analysis

When sampling matrices from the (finite) pool P, however, there is a non-zero probability that the same matrix is drawn multiple times when sampling is performed with replacement, i.e., there may be repeated indices in α(l) Such matrix repetitions, in general, will destroy the unbiased nature of our estimator Critically, the bias originates from the finite size of the pool from which we sample X(j) and would not arise for on-the-fly calculations and is thus an artifact of the bootstrapped approach A notable exception is when the X matrices belong to a zero mean Gaussian ensemble and each matrix product Zα is formed from the product of an odd number of matrices (N is odd). p. 2

Estimator Variance and Correlations

Additional fluctuations in Fˆ, however, can arise when different samples of Zα are, to some degree, correlated To assess the strength of these additional fluctuations, we again consider the simple case for which f(Z) = Z The total covariance C(K, M, N) hence characterises the degree to which Z(1,…,N) and Zα are correlated for a given permutation α Observing further that∥var[Z]∥1 = C(N, M, N), the relative magnitude of each contribution in the second term of Eq. (18) with respect to the first term is ρ/NZ, which decreases with matrix size M since K ≤ N. p. 5

Enumeration of Permuted Matrix Correlations

The frequency function ν(K, N) is peaked at a non-zero correlation, showing a non-monotonic dependence on K for a fixed N Most importantly, it is evident that the relative frequency νrel = ν(K, N)/N! of the higher correlation cases (i. p. 6e., smaller S = N − K) across the standard group becomes smaller for larger N. p. 6 Correlations thus decrease in both relative magnitude and relative frequency as the number of matrices in a given product increases and consequently the additional fluctuations introduced in an estimator Fˆ will also typically be lower. p. 6

Conclusion on Practicality

Our results show that in spite of the detrimental statistical effects of bootstrapping, practical application of either a single or dual pool approach is not limited by them Correlations that arise from matrix reuse, and the additional estimator covariance that follows, remain The consequences of spurious correlations will typically decrease in severity as the dimension of random matrices and the number of matrix products increases. Fortunately, bootstrapping approaches are only required in the regime where scattering channels are numerous and scattering media are many mean free paths in length. p. 8

Appendix A Details

The proof of Eq. (26) shows that for the identity permutation, S = 1 therefore completing the proof of Eq. p. 8 (26) The results of Table II and the analytic formulae for ν(K, N) highlight a number of important trends with respect to correlations between cascaded transfer matrices. p. 6 p. 6 We observe Eqs. p. 6 (28) and (29) correspond to integer sequences A005718 and A107991 respectively. p. 6 The results of Table II and the analytic formulae for ν(K, N) highlight a number of important trends with respect to correlations between cascaded transfer matrices. We note the following results (which have been numerically verified, via exhaustion, up to N = 13). p. 6 The frequency function ν(K, N) is never zero for all N. p. 6 The number of non-zero elements is M 2(N+1) for all possible permutations. p. 5

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Pool based bootstrapping for cascaded random matrix models can reduce both the number of matrix generations required and the number of matrix products compared to on-the-fly generation, enabling statistical studies of scattering in thick random media up to thicknesses of 30 mean free paths using fewer realizations.

  2. The system can estimate mean properties like transmission with reduced estimator bias by avoiding the introduction of additional estimator fluctuations that arise from matrix reuse, provided a sampling strategy is used that avoids repeated indices when sampling matrices from a finite pool P without replacement.

  3. The AI system can characterize the total variance of an estimate, denoted by var[Z]1, using the covariance term C(K, M, N), which quantifies the degree to which Z (1 to N) and Z alpha are correlated for a given permutation alpha.

  4. The system can determine the frequency ν(K, N) of various correlation strengths by enumerating the permutations across the standard group SN using combinatorial arguments derived from cycle structures in Eq. (26) through (29).

  5. For systems with many scattering modes and large media lengths, the AI system can utilize bootstrapping techniques because the relative frequency νrel = ν(K, N)/N! of the higher correlation cases (i.e., smaller S = N − K across the standard group becomes smaller for larger N.

Sources

Related papers