Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models
summary
The gist
Crypto agility maturity models aim to support organizations in preparing for cryptographic transitions, such as migrating to post-quantum cryptography <ref:2611.17003#pg6>.
In short
This research compared three industry-driven crypto agility maturity models against established design principles. The study found that while models vary based on their goals, they systematically lack measurable, reproducible assessment criteria and integrated guidance for improvement. This means current models are weak for informing real governance decisions regarding cryptographic transitions like post-quantum cryptography.
Key concepts
- Crypto Agility Maturity Models
- These are frameworks used to assess how prepared an organization is for cryptographic changes, such as migrating to new encryption standards (like post-quantum cryptography). The paper tested three different industry versions of these models to see how well they follow best practices for creating good maturity assessments.
- Design Principles for Maturity Models
- These are established rules used to evaluate whether a maturity model is well-designed. They focus on ensuring the model is clear, consistent, and provides actionable steps. The study used these principles as a benchmark to judge the three models being tested.
- Intersubjectively Verifiable Criteria
- This refers to assessment criteria that different assessors can agree upon consistently without bias or personal opinion. The paper found this was a major weakness across the models; if different people assess the same system, they might get different results because there is no clear, objective way to measure progress.
- Prescriptive Guidance
- This is the ability of a maturity model to tell users exactly what steps they need to take to improve their crypto agility. The analysis showed that none of the tested models provided this guidance directly within the model structure, making it hard for practitioners to know how to actually fix their systems.
Terminology used across episodes
This episode discusses
- Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models · Paper Radio
- Toward a Common Understanding of Cryptographic Agility -- A Systematic Review · Paper Radio
The paper
Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models · Read on arXiv
Leonie Wolf, Robin Leander Schröder, Sebastian Schinzel, Fabian Ising
Fraunhofer SIT · Fraunhofer Austria · National Research Center for Applied Cybersecurity ATHENE · HNFIZ Cybersecurity · FH Münster University of Applied Sciences
Crypto agility maturity models aim to support organizations in preparing for cryptographic transitions, such as migrating to post-quantum cryptography. Recognizing the importance of measurable maturity, NIST references several such models in its guidance. Given NIST's influence on cybersecurity practice, these models may shape organizational planning and governance mechanisms. However, their design quality has not been systematically evaluated. In this work, we assess three industry-driven maturity models against established design principles. Our analysis reveals recurring weaknesses, particularly the lack of measurable, reproducible assessment criteria and limited prescriptive guidance for improving crypto agility. At the same time, we highlight individual strengths that can assist practitioners in choosing among the models and derive concrete recommendations for strengthening future crypto agility maturity models.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models".
Elias: Here's the gist of the paper. Crypto agility maturity models aim to support organizations in preparing for cryptographic transitions, such as migrating to post-quantum cryptography <ref:2611.17003#pg6>.
Nadia: In this work, we assess three industry-driven maturity models against established design principles <ref:2611.17003#pg9>.
Elias: Our analysis reveals recurring weaknesses, particularly the lack of measurable, reproducible assessment criteria and limited prescriptive guidance for improving crypto agility <ref:2611.17003#pg10>.
Nadia: Pruning redundant reasoning tokens The paper conducts the first comparative evaluation of three NIST-cited industry-driven models and compares their design profiles to the existing CAMM evaluation.
Title and authors: Nadia: So this paper is called "Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models." It’s looking at these three industry-driven models that NIST actually mentions in their guidance for things like post-quantum cryptography migration.
Elias: Yeah, it’s basically taking those models and testing them against some established design principles. The main point is that we need to look closely at how these models are actually built before we let them shape how big organizations plan their security transitions.
Nadia: So, what does the paper actually find when it tests these three models? Are they all equally bad, or are some of them worse than others?
Elias: The analysis shows that they all have some weaknesses, but there are certain flaws that show up across the board. Most critically, the paper points out a big problem with how they measure things: there is a lack of criteria that people can actually agree on when they look at the results.
Priya: So you’re saying if two different teams looked at these models, they wouldn't get the same assessment because the scoring isn't clearly defined? That makes sense because measurement needs to be solid before we trust any maturity score.
Nadia: Exactly. And it seems like the biggest issue is that they don't provide clear instructions on how to actually improve a model once you get a result. It’s descriptive, but not really prescriptive enough for real-world action.
Elias: That’s right, they fail the prescriptive design principles from Pöppelbuß and Röglinger, which are meant to guide actual improvements in maturity. The paper says none of the three models fully satisfy those improvement requirements.
The paper's summary: Nadia: So if we boil down what this paper is saying, it’s that these maturity models exist to help organizations transition to new cryptography, like post-quantum crypto, but they aren't all built with the same quality standards.
Elias: Right. The researchers assessed three models—FS-ISAC, ATIS, and Deloitte—and compared them against Design Principles for maturity models. They found that while the models are different in their basic goals, they systematically fail to deliver on two key areas: having assessment criteria that everyone can agree on, and giving clear advice on how to get better.
Priya: I’m interested in the specifics of what they tested. How did they actually check if something was intersubjectively verifiable? That sounds like a real headache because agreement is hard in security assessments.
Nadia: They used those established design principles, specifically the Basic Design Principles, Descriptive Design Principles, and Prescriptive Design Principles that Pöppelbuß and Röglinger proposed. The focus there is on whether the assessment process itself is well-defined and if the results are verifiable across different assessors.
Elias: The paper points out a systematic failure in verifiability for both FS-ISAC and Deloitte because their assessment criteria aren't things you can check and agree on without a lot of subjective judgment. ATIS did better on the descriptive side, mostly because five of its twenty-three metrics gave a clear scale from one to ten.
Priya: So ATIS is the standout in terms of how much it describes what’s happening, but that doesn't mean it’s perfect if you can’t actually get a consistent score on it. We need data that reflects reality, not just theoretical scales.
Nadia: Right. And another thing they found was that none of the models integrated actual recommendations into their own structure to show you how to move forward, which is a big gap for practitioners trying to implement something right away.
The paper's improvements: Elias: Now the authors offer some suggestions on how we can use this analysis. They suggest that choosing a maturity model shouldn't just be about what it looks like on paper, but really matching the model’s basic principles to exactly what you are trying to achieve.
Nadia: So, for instance, if your main users are already using the NIST Cybersecurity Framework, then integrating something like Deloitte might be easier for your governance structure than picking one of these other models.
Priya: That makes sense from a practical standpoint. You want something that fits the existing ecosystem of tools you already have in place so you don't introduce too much friction into the process. But what about users who are focused purely on performance?
Elias: For people who are comfortable with key performance indicators, ATIS is suggested because it’s the most fine-grained model available for that, even though they noted it doesn't give clear maturity levels to communicate up to leadership.
Nadia: The paper suggests that FS-ISAC can actually work for two different groups at once: people who need a solid theoretical foundation and those who are already familiar with crypto agility but need it for different reasons. It seems like model choice really depends on the user’s background and the purpose of the assessment.
Priya: I wonder what they mean by that "primary users’ backgrounds." Are we talking about IT operations folks, or maybe procurement managers deciding where to spend money? That distinction is important for picking a model.
Elias: It seems it’s about matching the basic principles to the intended purpose, you know, figuring out if you need a descriptive tool or a prescriptive tool before you even look at the models themselves.
Conclusion: Nadia: So wrapping up this discussion on "Maturity squared: Assessing the Maturity of Crypto Agility Maturity Models," we see that these models are useful starting points, but they aren't ready to be the final word for how organizations should manage crypto agility.
Elias: That’s right. The main finding is that they consistently lack those measurable, reproducible assessment criteria and integrated guidance for actual improvement paths. If you need a model to inform real governance or procurement decisions, this is where they fall short right now.
Priya: From my side, the key thing for future work needs to be making sure that whatever we build next has solid measurability so different assessors can actually reach the same verdict. That’s what we need before we trust these scores in a real security context.
Nadia: I agree. And the paper’s core recommendation is that any model meant to guide governance—like setting procurement requirements or large-scale PQC migration strategies—needs to include explicit decision logic that links the maturity score directly to concrete steps for improvement.
Elias: Exactly. It has to move beyond just describing a state of maturity and show you exactly what action you need to take next, with clear paths forward. That’s the direction we need to head in for future model development.
Priya: I think focusing on that explicit decision logic is where the real value will be, because it moves us from theory into actual implementation planning.
Nadia: So, so if you're driving a security strategy right now, this paper gives you a good map of what’s missing in these current maturity models and how to steer toward something more usable. That’s all for this one on crypto agility models.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18