Breaking via Adaptive Neuron Clustering in Cryptanalytic Model Extraction

arXiv:2611.07370 · cs.CR · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Breaking via Adaptive Neuron Clustering in Cryptanalytic Model Extraction".

Elias: Here's the gist of the paper. Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence The single most informative sentence about the paper's subject and finding is SimBreak, an adaptive clustering rule that breaks the weight-similarity defense of TraintoDefend.

Nadia: The Threat Landscape Cryptanalytic model extraction represents a severe threat to deep learning models by drawing on cryptanalysis principles to design informative queries, thereby recovering hidden parameters with high precision and reconstructing functionally equivalent models Advances in this field have broadened the range of models and interfaces exposed to cryptanalytic extraction, motivating defenses that account for increasingly capable adversaries.

Elias: TraintoDefend was introduced as the first dedicated defense against cryptanalytic extraction by modifying the training objective to encourage different neurons to learn similar weights so that they switch activation states together.

Nadia: The Failure Mechanism of TraintoDefend The defense in TraintoDefend rests on the premise that similar weights cause neurons to switch activation states together, mixing their contributions to local derivative measurements and hindering signature recovery However, the original extraction attack can merge signatures from distinct neurons when sufficiently many normalized weight coordinates agree, even if the remaining coordinates differ substantially.

Paper summary: Nadia: We talked about the core idea of this paper, "Breaking via Adaptive Neuron Clustering in Cryptanalytic Model Extraction." The main point is that cryptanalytic model extraction is a serious threat because it allows people to recover hidden model parameters with high precision and guarantees that the recovered model acts functionally like the original one.

Elias: And to stop this, they looked at TraintoDefend, which aimed to make neurons learn similar weights so they switch their activation states together, making them harder to distinguish individually. But the paper finds a flaw in that defense structure.

Priya: What is that flaw specifically? Why can't neurons with similar weights always be distinguished by switching their activation states?

Nadia: The researchers noticed that neurons with similar weights do not necessarily switch activation states together when you make small changes to the model, which means the original extraction attack can still merge signatures from distinct neurons.

Elias: This merging of signatures is what they call signature clustering, and it’s a concrete obstacle that TraintoDefend introduces into the existing extraction attack. The original rule can combine signatures from different neurons even if some of their weight coordinates are very different.

Priya: So, this clustering is a technical problem that the paper identifies as a weakness in how TraintoDefend handles neuron similarity.

Nadia: Precisely, and they then introduce SimBreak to solve this clustering obstacle. SimBreak works by formulating pairwise signature matching as a test of whether two local observations agree under a common relative scale.

Elias: The key part of that is how it estimates the relative scale, which they figure out by least squares over all jointly observed weight coordinates, which is described in Section four point one <ref:2611.07370#pg1>.

Priya: So the method involves estimating this scale using a least squares approach across all the weights to check for agreement between local measurements. That’s a specific mathematical mechanism they are using to overcome the issue of clustering signatures.

Nadia: And they show that this results in a normalized residual that incorporates discrepancies across every single jointly observed coordinate, which prevents agreement on just a small subset of coordinates from being enough to pass the test.

Elias: It really forces the two local observations to agree based on a much broader set of data than what was previously used, which is exactly what breaks the clustering mechanism.

Priya: So they are essentially making the required agreement condition much stricter by demanding consistency across all observed weight coordinates simultaneously. That sounds like a solid way to handle noisy data in this context.

Conclusion: Nadia: Wrapping up, the paper "Breaking via Adaptive Neuron Clustering in Cryptanalytic Model Extraction" shows that TraintoDefend’s defense is not as strong as it appeared against adaptive attacks. SimBreak successfully breaks the layer signature recovery barrier by raising recovery from zero percent to one hundred percent across all six tested models.

Elias: The authors conclude that this finding challenges the premise of TraintoDefend, which was that weight similarity automatically prevents cryptanalytic extraction by forcing neurons to switch states together.

Priya: So what does this mean for the practical security landscape right now? Does it suggest we need to rethink how we design defenses against these kinds of adaptive attacks?

Nadia: It suggests that defenses based on neuron similarity must be more sophisticated; they have to account for attacks that exploit both weight directions and affine offsets, not just the simple coordinate matching assumptions used in existing implementations.

Elias: That’s the big picture—defenses need to adapt to how adversaries design their queries rather than just relying on a single defensive mechanism that seems robust.

Priya: So, for someone listening who isn't deep into the math, this paper tells us that you can't rely on simple similarity checks alone when trying to secure deep learning models from sophisticated extraction attacks.

Nadia: That’s right. The work demonstrates that adaptive attacks can exploit subtle discrepancies in how neurons operate even when they look similar on the surface.

Haojie Wu, Haolin Liu, Tianshuo Cong, Guoyan Zhang

School of Cryptologic Science and Engineering, Shandong University · School of Computer Science, Shanghai Jiao Tong University

cs.CR

Submitted: 2026-11-05

Updated: 2026-11-05

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence The…

Key concepts

Cryptanalytic Model Extraction
This is a method where an attacker uses mathematical principles, similar to cryptography, to query a neural network. The goal is to recover the hidden internal parameters (weights) of the model with high accuracy, even if those parameters are secret.
TraintoDefend
This was an initial defense mechanism designed against extraction attacks. It worked by training the model so that neurons learning similar weights would also tend to switch their activation states together. The idea was that this similarity would confuse the signature recovery process.
SimBreak
This is a new adaptive clustering rule used to counter TraintoDefend. Instead of relying on simple weight similarity, SimBreak estimates a consistent relative scale across all observed weight coordinates and uses that scale to test for both weight direction and bias consistency. This breaks the defense's assumption.
Signature Clustering
This is the phenomenon where an extraction attack merges the signatures (information about weights) from different neurons because their weight coordinates agree under a certain condition. TraintoDefend tried to prevent this, but SimBreak exploits this clustering by finding a common scale that allows these distinct signatures to be matched.

Terminology

Summary

Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence The single most informative sentence about the paper's subject and finding is SimBreak, an adaptive clustering rule that breaks the weight-similarity defense of TraintoDefend.

The Threat Landscape

Cryptanalytic model extraction represents a severe threat to deep learning models by drawing on cryptanalysis principles to design informative queries, thereby recovering hidden parameters with high precision and reconstructing functionally equivalent models Advances in this field have broadened the range of models and interfaces exposed to cryptanalytic extraction, motivating defenses that account for increasingly capable adversaries. TraintoDefend was introduced as the first dedicated defense against cryptanalytic extraction by modifying the training objective to encourage different neurons to learn similar weights so that they switch activation states together. p. 1

Where the paper says it

To counter this threat, TraintoDefend [12] was introduced as the first dedicated defense against cryptanalytic extraction, making its robustness a central question for the protection of model parameters. TraintoDefend modifies the training objective to encourage different neurons to learn similar weights.

The Failure Mechanism of TraintoDefend

The defense in TraintoDefend rests on the premise that similar weights cause neurons to switch activation states together, mixing their contributions to local derivative measurements and hindering signature recovery However, the original extraction attack can merge signatures from distinct neurons when sufficiently many normalized weight coordinates agree, even if the remaining coordinates differ substantially. This signature clustering is a concrete obstacle introduced by TraintoDefend in the existing extraction attack. p. 2

Where the paper says it

Joint switching depends on the probe center, direction, and step size, so weight similarity alone does not ensure that a probe crosses multiple activation boundaries. We identify signature clustering as a concrete obstacle introduced by TraintoDefend in the existing extraction attack.

The SimBreak Solution

SimBreak is an adaptive clustering rule that breaks the weight-similarity defense of TraintoDefend It estimates a relative scale from all jointly observed weight coordinates and uses the same scale to test both weight direction and bias consistency. The rule formulates pairwise signature matching as a test of whether two local observations agree under a common relative scale It estimates this scale by least squares over all jointly observed weight coordinates, as described in Section 4. p. 2

Where the paper says it

To overcome the clustering obstacle introduced by TraintoDefend, SimBreak formulates pairwise signature matching as a test of whether two local observations agree under a common relative scale. It estimates this scale by least squares over all jointly observed weight coordinates, as described in Section 4.

1.

Experimental Validation

The experimental evaluation demonstrates that replacing the original rule with SimBreak raises first-layer signature recovery from 0% to 100% across all six models. This complete recovery is achieved across widths from 8 to 64 in two-hidden-layer networks and also in the deeper models, as shown in Section 5. p. 2

Where the paper says it

Replacing the original rule with SimBreak raises first-layer signature recovery from 0% to 100% across all six models, with a maximum reported relative recovery error of 2 −22. Complete recovery is achieved across widths from 8 to 64 in two-hidden-layer networks and also in the deeper models, as shown in Section 5.

4. The maximum reported relative recovery error is 2−22. p. 2
Where the paper says it

We evaluate SimBreak on six models with varying widths and depths and compare it with the original coordinate agreement rule. Replacing the original rule with SimBreak raises first-layer signature recovery from 0% to 100% across all six models, with a maximum reported relative recovery error of 2 −22.

27.

Conclusion and Implications

The work challenges TraintoDefend’s premise that weight similarity prevents cryptanalytic extraction by causing neurons to switch activation states together SimBreak bypasses the protected layer signature recovery barrier of TraintoDefend, raising recovery from 0% to 100% across all six evaluated models with a maximum reported relative error of 2−22.27. This underscores the need to evaluate defenses against adaptive attacks. p. 1

Where the paper says it

These advances broaden the range of models and interfaces exposed to cryptanalytic extraction, motivating defenses that account for increasingly capable adversaries. To counter this threat, TraintoDefend [12] was introduced as the first dedicated defense against cryptanalytic extraction, making its robustness a central question for the protection of model parameters.

The findings suggest that defenses based on neuron similarity must account for attacks that exploit both weight directions and affine offsets, rather than relying on the coordinate matching assumptions of an existing implementation.

References

Asselineau, R., Derbez, P., Fouque, P.A., Minaud, B.: Cryptanalytic extraction of deep neural networks with non-linear activations Canales-Martínez, I. p. 0

Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

A., Chávez-Saab, J., Hambitzer, A., Rodríguez-Henríquez, F., Satpute, N., Shamir, A.: Polynomial time cryptanalytic extraction of neural network models Carlini, N. p. 1
Where the paper says it

Cryptanalytic model extraction, by contrast, draws on cryptanalysis principles to design informative queries, thereby recovering hidden parameters with high precision and reconstructing functionally equivalent models, turning black-box access into model possession in effect. Since Carlini et al.

, Jagielski, M., Mironov, I.: Cryptanalytic extraction of neural network models Chen, Y. p. 0
Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

, Dong, X., Guo, J., Shen, Y., Wang, A., Wang, X.: Hard-label cryptanalytic extraction of neural network models Carlini, N. p. 7
Where the paper says it

However, their adaptive attack recovers models by estimating directional derivatives over larger probing intervals. Moreover, cryptanalytic extraction has also been demonstrated in the hard-label setting, where the attacker observes only predicted class labels [3, 5].

, Chávez-Saab, J., Hambitzer, A., Rodríguez-Henríquez, F., Shamir, A.: Polynomial time cryptanalytic extraction of deep neural networks in the hard-label setting Chen, Z. p. 7
Where the paper says it

However, their adaptive attack recovers models by estimating directional derivatives over larger probing intervals. Moreover, cryptanalytic extraction has also been demonstrated in the hard-label setting, where the attacker observes only predicted class labels [3, 5].

, Tang, S., Gao, Z., Su, Y., Qin, L., Dong, X.: Algebraic cryptanalytic extraction on hard-label neural networks Crankshaw, D. p. 7
Where the paper says it

However, their adaptive attack recovers models by estimating directional derivatives over larger probing intervals. Moreover, cryptanalytic extraction has also been demonstrated in the hard-label setting, where the attacker observes only predicted class labels [3, 5].

, Wang, X., Zhou, G., Franklin, M.J., Gonzalez, J.E., Stoica, I.: Clipper: A low-latency online prediction serving system (2017) Foerster, H. p. 5
Where the paper says it

These activations serve as known inputs to the next layer, allowing the attacker to repeat the above operations for the next layer [4, 16]. The process continues through the hidden layers, after which the final affine layer is recovered by solving a linear system using the recovered hidden activations and oracle outputs.

, Mullins, R., Shumailov, I., Hayes, J.: Beyond slow signs in highfidelity model extraction Gan, W. p. 3
Where the paper says it

2 Cryptanalytic Extraction Attacks Cryptanalytic extraction reconstructs fully connected ReLU networks by exploiting their piecewise affine structure through queries that return raw numerical outputs or hard labels [3–5]. The attack proceeds layer by layer, recovering neuron signatures and then determining their signs before moving to the next layer.

, Wan, S., Philip, S.Y.: Model-as-a-service (maas): A survey Ito, A. p. 0
Where the paper says it

Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how. To commercialize DNNs while preserving control over their parameters, providers typically offer them under the Model-as-a-Service (MaaS) arXiv:2611.

, Miura, T., Todo, Y.: End-to-end hard-label cryptanalytic model extraction using efficient sign recovery Jagielski, M. p. 1
Where the paper says it

[4] demonstrated how the piecewise affine structure of ReLU networks can be exploited to recover model parameters through output queries, cryptanalytic model extraction has advanced along several directions. Improvements in signature estimation and sign recovery have made extraction more efficient and enabled attacks on deeper networks [16].

, Carlini, N. p. 1
Where the paper says it

Cryptanalytic model extraction, by contrast, draws on cryptanalysis principles to design informative queries, thereby recovering hidden parameters with high precision and reconstructing functionally equivalent models, turning black-box access into model possession in effect. Since Carlini et al.

, Berthelot, D., Kurakin, A., Papernot, N.: High accuracy and high fidelity extraction of neural networks LeCun, Y. p. 1
Where the paper says it

Conventional learning-based extraction mainly approximates task behavior from sampled queries. Cryptanalytic model extraction, by contrast, draws on cryptanalysis principles to design informative queries, thereby recovering hidden parameters with high precision and reconstructing functionally equivalent models, turning black-box access into model possession in effect.

, Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition li, C. p. 1
Where the paper says it

Nevertheless, recent progress in cryptanalytic model extraction [2, 4, 5] represents a severe threat. Conventional learning-based extraction mainly approximates task behavior from sampled queries.

, Zhuang, L., Li, D., Tang, Y., Gong, Z.: Polynomial-time cryptanalytic extraction of graph neural networks in the hard-label setting Liu, H. p. 7
Where the paper says it

However, their adaptive attack recovers models by estimating directional derivatives over larger probing intervals. Moreover, cryptanalytic extraction has also been demonstrated in the hard-label setting, where the attacker observes only predicted class labels [3, 5].

, Siproudhis, A., Boura, C., Peyrin, T.: Model extraction of convolutional neural networks with max-pooling Liu, H. p. 0
Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

, Siproudhis, A., Experton, S., Lorenz, P., Boura, C., Peyrin, T.: Navigating the deep: End-to-end extraction on deep neural networks Noman, I. p. 0
Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

, Vukelic, M., Jasuja, V.: Output rounding is not a free defense against cryptanalytic neural network extraction Qi, X. p. 7
Where the paper says it

4 Other Defenses Noman et al [17]. investigate output rounding as a defense against cryptanalytic extraction, reducing the precision of returned outputs while leaving the model parameters unchanged.

, Lei, H., Wei, L., Sun, X., Wang, M.: Cryptanalytic extraction of neural networks with various activation functions Sun, X. p. 0
Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

, Lei, H., Wei, L., Qi, X., Hu, K., Wang, M.: Cryptanalytic extraction of convolutional neural networks Tang, S. p. 0
Where the paper says it

Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses. Keywords: Cryptanalytic Model Extraction · TraintoDefend · Adaptive Signature Clustering 1 Introduction Deep Neural Networks (DNNs) constitute valuable digital assets, embodying considerable investment in proprietary data, computational resources, and engineering know-how.

, Chen, Z., Su, Y., Gao, Z., Qin, L., Dong, X.: Normal alignment: Improved cryptanalytic sign recovery on hard-label networks Wang, S. p. 7
Where the paper says it

However, their adaptive attack recovers models by estimating directional derivatives over larger probing intervals. Moreover, cryptanalytic extraction has also been demonstrated in the hard-label setting, where the attacker observes only predicted class labels [3, 5].

, Lei, H., Wei, L., Liu, Q., Hu, K., Wang, M.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive defense mechanism for cryptanalytic model extraction: The improved system can employ SimBreak to distinguish neurons with similar weights by jointly evaluating weight direction and bias consistency under a relative scale fitted over all observed weight coordinates.

  2. Enhanced signature recovery capability: The improved system can achieve complete first layer signature recovery in models protected by TraintoDefend, raising recovery from 0% to 100% across all evaluated configurations with a maximum reported relative error of 2-22.27.

  3. Robust defense against adaptive attacks: The improved system resists adaptive extraction by challenging the use of neuron weight similarity as a sufficient basis for protection, demonstrating that adaptive evaluation is essential to assessing cryptanalytic extraction defenses.

Abstract

Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence. To counter this growing threat, (NeurIPS 2025) was proposed as the first dedicated defense against cryptanalytic extraction and remains the only defense believed to be effective to date. It encourages neurons to learn similar weights so that they switch activation states together, making their individual contributions harder to distinguish. However, we notice that neurons with similar weights need not switch activation states together under small perturbations. Instead, weight similarity can cause the original clustering rule to merge signatures from distinct neurons when only a subset of their normalized weight coordinates agree. Motivated by this observation, we introduce, an adaptive clustering rule that breaks the weight-similarity defense of. estimates a relative scale from all jointly observed weight coordinates and uses the same scale to test both weight direction and bias consistency. Across six protected fully connected networks trained on MNIST, raises target layer signature recovery from 0% to 100%, with a maximum relative recovery error of 2-22.27. Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses.

Related papers