Layer-wise Input Recoverability in Deep Neural Networks

arXiv:2610.14290 · cs.LG · Submitted 2026-10-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Layer-wise Input Recoverability in Deep Neural Networks".

Jane: The gist The inner workings of deep neural networks are still not well understood,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper today called "Layer-wise Input Recoverability in Deep Neural Networks." It seems like they're trying to figure out how much of what a convolutional layer actually sees can be rebuilt from what that layer spits out.

Jane: That sounds really specific, Tom. So instead of just looking at the whole network's output, they want to look at the individual patches as they go through each layer and see how well they can reconstruct their original input.

Tom: Exactly. The core idea here is defining recoverability as a fraction of variance in those input patches that a decoder can reconstruct from the layer’s output. They use this method specifically because it scales to really deep networks, even with large feature maps, by only probing the patches instead of the entire feature map.

Lu: What's interesting about their measurement is how they define it: they use a held-out R2 value, which is one minus mean squared error divided by variance of the input patches. But they admit that because their decoders are restricted, this measure is closer to a V-information style measure than true Shannon information (as seen in the paper) <ref:2610.14290#pg2>.

Meng: If we're talking about what that means for us on the ground, it suggests that some layers might be holding onto a lot of useful input detail while others are just throwing away information as they go deeper. That’s something we worry about when deploying models at scale.

Lalam: I see. So they're essentially checking the information bottleneck effect layer by layer, focusing on local input patches instead of the global feature map, which lets them study this mechanism even in deep CNNs like VGG or ResNet architectures where scaling mutual information estimators was a problem before (as seen in Cheng et al., two thousand eighteen) <ref:2610.14290#pg3>.

Tom: Right. Then they set up three baseline encoders for every layer: a pretrained one, a random one, and an end-to-end one trained with the decoder. They also introduced learning-attributed recoverability, which is basically the difference between the pretrained and random recoveries to separate how much of that performance comes from learning versus just what's in the architecture itself.

Jane: That separation is key. So they are trying to isolate what training actually changes by comparing those three settings, and they even introduced single-property random controls, like transferring singular vectors or activation sparsity to a random encoder.

Lu: Which is interesting because they found some consistent trends across different architectures and layers. One major finding is that recoverability generally decreases as you go deeper into the network across different models. But they also saw exceptions, like MobileNetV2 showing a less pronounced decrease in recoverability over depth, moving from zero point nine nine down to zero point six six (as seen on page two).

Meng: And they found a link between layer type and compression ratios, measured by the ratio d over Cout, which seems strongly correlated with effective compression. If effective compression is tied so closely to recoverability, it means we should probably be designing our layers around that compression ratio, not just the nominal one.

Jane: They also looked at why the learning-attributed gaps exist. They found that copying singular vectors tends to raise recoverability while copying the singular-value spectrum or activation sparsity tends to lower it. It’s not a single property driving everything across all models, which is something they flag as a limitation, though some specific matches worked well, like activation-sparsity matching in AlexNet and SqueezeNet (as mentioned on page three).

Tom: The authors also pointed out that recoverability is decoder-relative. If you use a wider decoder, you can get better results, so the values they report are actually lower bounds on what a stronger decoder could achieve. That’s an important caveat for anyone trying to benchmark these numbers.

The paper's summary: Tom: So we're talking about how much of a layer’s input data you can actually pull back out after that layer has processed it, and this paper is looking at that layer by layer across these deep neural networks.

Jane: Right, so it basically asks this question to the AI—how much information about what went into a specific convolutional filter can we reconstruct from just the output of that filter? It defines it as the fraction of variance in those input patches that a decoder can actually rebuild.

Tom: Exactly. They’re not looking at the whole feature map, they are sampling small patches from each layer's input and trying to see what happens with those specific pieces. It’s scalable for deep networks because you don't have to deal with the massive full feature map all at once.

Lu: What they introduce is this R2 score for recoverability, which is one minus mean squared error divided by the variance of those input patches. They’re being careful here, though, and they admit this isn't Shannon information. It’s a decoder-relative measure because if you use a wider reconstruction probe, you get a better number <ref:2610.14290#pg2>.

Meng: That matters for deployment because it means we have to be careful about what tools we use to try and interpret the model's internal state. If the decoder is too simple, our recoverability score will just look artificially low regardless of how good the layer actually is at retaining information.

Jane: And they tested this by comparing three ways of encoding those layers: a pretrained encoder, a totally random one, and one trained end-to-end with a decoder. This helps them see if the information retention comes from the training itself or just the specific architecture of that layer.

Tom: They also introduce learning-attributed recoverability as the difference between those pretrained and random scores. That’s their way of trying to separate what's due to learning from what’s just due to having a certain type of layer built into the network structure.

Lu: And they found some interesting trends in that comparison across different models. For instance, they saw recoverability generally going down as you go deeper into the network, but some models like MobileNetV2 actually showed less drop than others <ref:2610.14290#pg2>.

Meng: They also connected this recoverability directly to how compressed the layer is. They found that effective compression is strongly correlated with recoverability across all tested models <ref:2610.14290#pg3>.

Jane: And they also looked at what specific properties of the pretrained weights cause those gaps between learning and architecture. Copying singular vectors tends to raise recoverability, while copying the singular-value spectrum or activation sparsity tends to lower it.

Tom: So, in practical terms for us, it tells us that if we want to boost layer-wise input recoverability during training, we should look at what's driving those specific properties. We can try to transfer things like singular vectors or BatchNorm parameters from a pretrained model onto a random encoder to see if it helps.

Jane: That’s the implication for building better representations—we can use these controls to target the exact property that boosts information retention in a given architecture, rather than just hoping the end-to-end training gets us there.

Lu: But they also pointed out their limitation here. Recoverability is dependent on the decoder you choose. So, those numbers are really just lower bounds; a better reconstruction probe could find more variance than what they measured <ref:2610.14290#pg2>.

Meng: That means we can't just trust a single recoverability number to judge a layer’s quality; we have to consider the capacity of the tools we use to measure it, which is another layer of complexity for production systems.

Jane: So, while this study gives us concrete ways to probe and control information flow within deep networks, they suggest that for the next step, we might need to look at patch-level Shannon estimators instead of just this V-information style measure for a more complete picture.

The paper's improvements: Tom: Okay, so we've seen how they measure recoverability—the fraction of variance you can get back from a layer's input—and now we’re looking at what they suggest to fix things or make the method better.

Jane: So what are the actual improvements they propose? What do they think is missing in this current approach, and how do we fix it?

Lu: They really push two big ideas here. First, adapting training strategies by using those single-property random controls we talked about earlier. They want to use them to maximize layer-wise input recoverability because each control actually changes the number in a consistent direction across different models <ref:2610.14290#pg3>.

Meng: If we can use those properties—like copying singular vectors—to specifically boost the information retained from each convolutional layer, that could be really useful for production deployment, right? It lets us fine-tune initialization or regularization to get better local information retention.

Tom: Exactly. That’s a direct lever for model tuning. Second improvement is targeting network architecture design by focusing on effective compression. They found that this metric is strongly correlated with recoverability, so engineers should try to make sure the effective compression stays near one no matter how deep the network gets <ref:2610.14290#pg3>.

Jane: That means we shouldn't just look at the nominal compression ratio, but figure out how that affects information retention across all layers, which is a much more practical design goal.

Lu: And third, they point out that since recoverability depends on the decoder you pick, we should recognize that these scores are lower bounds. This means if you use a wider reconstruction probe later on, you will get higher results than what’s reported here <ref:2610.14290#pg2>.

Meng: That's a good warning for anyone trying to benchmark this stuff; you need to account for your probe's capacity when interpreting the numbers. It prevents us from overestimating how much information we actually have.

Tom: And they also suggest that since patch-level probing works on deep networks, we should use Shannon information estimators at that level instead of just sticking with the V-information style measure for a tighter connection to the info bottleneck framework.

Jane: So the future direction is to move toward using those more robust Shannon estimators at the patch level so our analysis aligns better with established information theory.

Lu: This helps bridge that gap between their current measurement style and what’s theoretically expected from information retention in deep models.

Conclusion: Tom: So to wrap things up, this paper "Layer-wise Input Recoverability in Deep Neural Networks" shows that we can actually probe how much of a layer's original input data stays recoverable after that layer processes it, and they found that this recovers information depending on the specific weights and properties of the model.

Jane: Right. The core finding is that you can measure this at every layer, not just once at the end, and it tells us exactly how much detail is retained throughout a deep network's journey.

Lu: They showed that effective compression is strongly linked to this recoverability score, which means designing layers around their compression ratios actually matters for information retention.

Meng: For deployment, that correlation between compression and recoverability gives us a concrete metric to target when we want to ensure critical bottleneck characteristics are preserved across the entire network depth.

Jane: They also showed how training effects can be separated from architecture itself using learning-attributed recoverability, which helps diagnose precisely what properties of the weights—like singular vectors—are actually boosting that retention during training.

Lalam: From my side, I’m confident that this analysis is solid; it gives us a clear roadmap for understanding the information bottleneck in deep models <ref:2610.14290#pg1>.

Tom: It’s a useful tool for engineers looking to tune initialization or regularization specifically to boost the information retained from each convolutional layer.

Jane: And they did flag that their measurement is decoder-relative, meaning wider probes can improve the results, so we have to keep that caveat in mind when using these scores.

Lu: They also suggested future work should look at Shannon information estimators at the patch level for a more complete picture aligned with information bottleneck theory.

Meng: That’s a solid direction; if we can make those patch-level estimators efficient, it could give us much more granular insight into why certain layers are losing data.

Lalam: I’m sure this analysis is solid; it gives us a clear roadmap for understanding the information bottleneck in deep models <ref:2610.14290#pg1>.

Max Spiridonov, Alireza Rahmansetayesh, Pierre Baldi

University of California, Irvine

cs.LG

Submitted: 2026-10-09

Updated: 2026-10-09

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 81/100

The gist: The gist The inner workings of deep neural networks are still not well understood, and this study investigates how much information about a layer's own input can be recovered from its output in

Key concepts

Layer-wise Input Recoverability: This measures how much of the original input patch variance a decoder can reconstruct using only the representations (outputs) from a specific convolutional layer. It is calculated as the coefficient of determination (R2), representing the fraction of input variance that can be recovered, constrained by the decoder's complexity.
Probing and Controls: The study compares three encoder setups: pretrained (using fixed weights), random initialization, and end-to-end training. Learning-attributed recoverability separates the effect of learning from architecture by comparing pretrained versus random encoders. Specific controls test how transferring properties like singular vectors or activation sparsity affects this gap.
Effective Compression: This refers to the layer's compression ratio, defined as $d/C_{out}$, where $d$ is the dimension of the input patch vector and $C_{out}$ is the number of output channels. The research finds a strong correlation between this compression ratio and how much information can be recovered from that layer's output.
Decoder-Relative Measure: Recoverability is dependent on the decoder used for reconstruction. Because the decoders tested are restricted, the reported values represent lower bounds on what a more powerful decoder could achieve. This means the measured recoverability is relative to the specific reconstruction method employed.

Summary

The gist The inner workings of deep neural networks are still not well understood, and this study investigates how much information about a layer's own input can be recovered from its output in convolutional neural networks (CNNs)

Layer-wise Input Recoverability

This research introduces layer-wise input recoverability, defined as the fraction of variance in the patches a convolutional layer reads that a decoder can reconstruct from the layer’s output. The method probes each convolutional layer on its input patches instead of the full feature map, which allows it to scale to deep CNNs with large feature maps. Recoverability is measured as the held-out R2 = 1 − MSE(X, Xˆ)/Var(X), using a linear decoder and a linear decoder with a residual MLP of width m · Cout. Because the decoders are restricted, this measure is closer to V-information (Xu et al., 2020) than to Shannon information.

Probing and Controls

The study compares three baseline encoders for each layer: pretrained (pre): the encoder is initialized and fixed with pretrained weights, random (rand): the encoder is initialized with the framework’s default random initialization and fixed, and end-to-end (e2e): the encoder is initialized randomly and trained jointly with the decoder to minimize reconstruction MSE. Learning-attributed recoverability is defined as R2 pre − R2 rand, which serves to separate the effect of learning from architecture. To isolate what training changes, single-property random controls are introduced, which transfer one property of the pretrained weights onto a random encoder such as singular vectors, singular-value spectrum, activation sparsity, or BatchNorm parameters.

Key Findings on Recoverability Trends

The analysis reveals several consistent trends across the tested architectures and layers. First, recoverability generally decreases over depth across architectures, with MobileNetV2 exhibiting a less pronounced decrease in recoverability over depth (from 0.99 to 0.66) and EfficientNet-B0’s last layer recoverability rising to 0.79 from the previous layer’s 0.49. Second, a layer’s type is set by Cin, Cout, and K, and these also determine its nominal compression ratio d/Cout = Cin · K2/Cout. The study finds that effective compression is strongly correlated with recoverability, with Spearman ρ ranging from −0.55 to − 0.91 across models.

Properties Driving Learning-Attributed Gaps

The comparison between pretrained and random encoders shows that learning-attributed recoverability is not uniform across architectures. The study identifies specific properties that influence this gap, noting that copying the pretrained singular vectors tends to raise recoverability, while copying the pretrained singular-value spectrum or activation sparsity lowers it. While no single control explains the gap across all models, some controls are highly effective in specific contexts; for instance, activation-sparsity matching in AlexNet and SqueezeNet and BatchNorm transfer in DenseNet121 and EfficientNet-B0 account for the gap.

Limitations and Future Directions

The paper notes that Recoverability is decoder-relative, meaning wider decoders can improve results, so the values reported are lower bounds on what a stronger decoder could recover. Furthermore, the random controls demonstrate how properties are associated with recoverability rather than being direct causes, and they do not account for interactions between properties. The authors encourage future work to develop more random controls that account for different properties of pretrained weights. Finally, because patch-level probing scales to deep CNNs, the authors suggest using Shannon information estimators at the patch level to align the analysis more closely with the information bottleneck framework.

--- Page 1 ---

LAYER-WISE INPUT RECOVERABILITY IN DEEP NEURAL NETWORKS

Max Spiridonov

Department of Computer Science

University of California, Irvine

mspirido@uci.edu

--- Page 2 ---

Other work in representation visualization asks what a representation learns by mapping it back to the pixel domain, with deconvolutional, optimization-based, or up-convolutional inversions of deep features (Zeiler & Fergus, 2013; Mahendran & Vedaldi, 2014; Dosovitskiy & Brox, 2016). Probes trained to predict labels from a layer’s representations (Alain & Bengio, 2018) avoid reconstruction, but their results reflect the probe’s capacity as well as the quality of the representations (Belinkov, 2021).

--- Page 3 ---

Shwartz-Ziv & Tishby (2017) took this framework and applied it to small (up to 7 layer) fullyconnected neural networks on a simple binary classification task. They found two distinct training phases: a fitting phase where the information I(Xˆ; Y) increases, and a longer compression phase where the information I(X; Xˆ) decreases.

--- Page 4 ---

Figure 1: Method overview. A. Patches are sampled from each probed layer’s input feature map (Section 3.1). B. Each layer’s patches are scored under three encodings of the same architecture: 1) the network’s pretrained encoder, 2) a random encoder, and 3) an encoder trained jointly with the decoder. All decoders are trained to minimize reconstruction mean squared error (MSE).

--- Page 5 ---

3 METHOD

3.1 SAMPLING INPUT PATCHES

We first perform a forward pass over our CNN. At each convolutional layer, we sample spatial positions from its input feature map (or image for the first layer). At each sampled position we extract the K × K, Cin-channel patch the layer’s filters read, flatten it to a vector x ∈ R d, where d = Cin · K2, and collect these patches into a tensor X ∈ R N×d per layer, where N is the number of patches sampled. We used N = 40,000 train and 8,000 validation images with 1 patch per image for our experiments, with ablations on number of images and patches per image in Appendix A.1, where recoverability changes by at most 0.0104 and learning-attributed recoverability by at most 0.002.

--- Page 6 ---

3.2 RECONSTRUCTION PROBE

For each layer’s input patches, we first find their representations z. Our encoder is a fully-connected layer, followed by the layer’s batch normalization and activation function. Thus, the representation z of an input patch is exactly equal to the convolutional layer’s output at that location. Now, our goal is to probe how recoverable a layer’s input patches are given their representations. We fit a decoder g (m) reconstructing layer m’s input X from the patch representations. Our simplest decoder is a single fully-connected layer with no activation function—Xˆ = Wlinz + blin. This allows us to perform convex optimization using linear least-squares, given patch representations z and reconstruction targets X. Our more expressive decoder adds a small nonlinear residual to the linear map. For a patch representation z ∈ R Cout and an expansion factor m ∈ 1, 4, 16, Xˆ = g (m) (z) = Wlin z + blin z linear part + W2 σ(W1z + b1) + b2 residual MLP, (1) where W1 ∈ R mCout×Cout, W2 ∈ R d×mCout, and σ is the GELU activation function.

--- Page 7 ---

3.3 DEFINITION OF RECOVERABILITY

Recoverability is simply the coefficient of determination, R2 = 1− MSE(X, Xˆ)/Var(X) for input patches X and the reconstructed patches, Xˆ, measured on validation set patches after probe training. Var(X) is the variance of the input patches X. Recoverability is thus the fraction of variance of a layer’s input patches that can be reconstructed from their representations. Because our decoders are restricted, recoverability is a decoder-relative, V-information-style measure and not Shannon information.

--- Page 8 ---

At each layer, we find the recoverability of input patches using the layer’s pretrained weights. However, some recoverability is forced by architecture. For example, the commonly used ReLU activation function is non-injective, resulting in variance that is unable to be recovered. Layers with a higher compression ratio, measured by d/Cout, have a tighter bottleneck which also results in less recoverability. To assess how pretrained weights affect recoverability, we compare the pretrained encoder against controls. Each control changes only the encoder weights of the probed layer; the input patches are identical across conditions, and the decoder is always optimized to minimize reconstruction MSE. We use three baseline encoders: 1.

Improvements for AI systems

Adapt model training strategies by incorporating single-property random controls to maximize layer-wise input recoverability, as each control changes recoverability with a consistent sign across models. This allows for targeted weight regularization or initialization adjustments that specifically boost the information retained from each convolutional layer.
Optimize network architecture design by targeting effective compression, as effective compression is strongly correlated with recoverability and it explains anomalies where nominal compression fails. Engineers can design layers where effective compression below 1 sit near R2 ≈ 1 no matter their depth, ensuring that critical information bottleneck characteristics are preserved across the network's depth.
Develop layer-specific pre-training protocols by utilizing learning-attributed recoverability to diagnose architectural strengths, as learning-attributed recoverability is not uniform across architectures. This enables researchers to know precisely which properties—such as copying the pretrained singular vectors tends to raise recoverability—are beneficial for a specific network structure.
Improve model robustness against structural changes by understanding decoder capacity limits, recognizing that recoverability is decoder-relative: in layers with a real bottleneck, wider decoders keep improving it, so our values are lower bounds. This informs the selection of appropriate reconstruction probes to ensure that the measured information retention reflects the layer's inherent capacity rather than probe limitations.