Taxonomy-Driven Analysis of Open-Source AI Risk Mitigation Tools

The paper, authored by Afreen Alam, Evgenija Popchanovska, Ana Gjorgjevikj, Maryan Rizinski, Lubomir T. Chitkushev, Irena Vodenska, and Dimitar Trajanov, addresses the challenge of mapping open-source AI risk mitigation tools to a structured risk taxonomy. The central problem is that "the current tooling landscape remains fragmented. Tools are typically built for specific engineering use cases and described in technical terms that do not align cleanly with governance frameworks or risk taxonomies, creating uncertainty about which tools address which risks, where capabilities overlap, and where critical gaps remain."

The paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools. Specifically, the authors systematically map the technical capabilities of 21 prominent open-source tools to the 32 sub-categories of the extended MIT AI Risk Mitigation and Response Taxonomy using an LLM-assisted retrieval-augmented generation pipeline... to analyze the relevant source code and documentation for each tool and extract relevant capabilities mapping to each taxonomy category.

The paper situates its work in the context of rapid enterprise adoption of LLMs, noting that LLMs generate probabilistic results, they can produce inaccurate or unsupported claims, expose sensitive information, respond unpredictably to adversarial inputs, or reproduce biased patterns embedded in training data. These risks are described as "especially consequential in regulated sectors such as financial services, insurance and legal services, where system failures can lead to compliance violations, financial loss, reputational harm, or adverse effects on individuals."

The authors argue that Manual review processes may be appropriate during early experimentation, but are rarely sufficient for production systems that process large volumes of user interactions in real time. They identify a literature gap: "relatively little work that systematically maps concrete tool capabilities to enterprise risk categories or to lifecycle-wide mitigation workflows. In particular, there is limited guidance on how organizations should select and compose open-source evaluation and security tools to cover specific risk subcategories, how to identify coverage gaps, and how to connect these technical assets to continuous feedback loops that include human oversight and align with sectoral frameworks such as the FS-AI RMF."

The study examined 21 open-source tools, including frameworks such as Promptfoo, DeepEval, Garak, PyRIT, NeMo Guardrails, and Langfuse. Tools were selected based on a combination of adoption and maintenance indicators, including GitHub activity, number of stars and forks, recent commit history, documentation quality, and relevance to enterprise AI risk mitigation. Four inclusion criteria were applied: (1) open-source availability and licensing, (2) community adoption and maturity, (3) active maintenance, and (4) functional relevance to LLM evaluation, runtime guardrails, adversarial red-teaming, or system observability. An initial pool of 28 candidates was reduced to 21, with exclusions reflecting misalignment with the study's focus on mature, auditably documented, enterprise-relevant mitigation capabilities.

The 21 tools are grouped into four functional clusters: "(i) evaluation and red-teaming frameworks (Promptfoo, DeepEval, RAGAS, PyRIT, Garak, Giskard), (ii) runtime guardrails and content-safety systems (NeMo Guardrails, LLM Guard, Rebuff, WildGuard, OpenGuardrails), (iii) observability and monitoring platforms (Langfuse, OpenLIT, Arize Phoenix, Evidently), and (iv) security, privacy, and robustness libraries (Adversarial Robustness Toolbox, ModelScan, Diffprivlib, PrivacyRaven, Holistic AI, with Innodata providing curated datasets and benchmarking support rather than direct runtime controls)."

A three-prompt chain was employed, balancing scalability, interpretability, and control over hallucination. The prompt design evolved through iterations: an initial 'all-at-once' batch extraction... proved unreliable due to context-window overload and superficial incomplete mappings, leading to a Single-Tool Anchor and Extract strategy. Key elements included a strict auditor persona and sequential category-wise evaluation... with explicit recording of true negatives ('None Found'), an aggressive noise filtering approach, a 'zero-inference' rule that prohibited capability assumptions based on marketing or high-level claims, and mandatory citations to specific code artefacts for every positive mapping.

Phase 1 involved repository ingestion using GitIngest (data ingested as of March 2026) and analysis via Google's NotebookLM, selected for its native support for large, multi-document corpora and its underlying Gemini models offer a substantially large 1-million token context window. The authors acknowledge a reproducibility limitation: "NotebookLM is a closed commercial product whose internal retrieval, chunking, and model versions are not fully user-controllable and may evolve over time, constraining exact replication of this pipeline by other researchers."

The extraction distinguished explicit mappings (direct implementation) from derived mappings (where a mitigation effect reasonably followed an explicit technical capability, even when the tool did not use terminology consistent with the taxonomy). Phase 2 involved matrix synthesis with a deterministic left-join rule, using the taxonomy CSV as the master row template and the 21 tools as the master column template, split into four parts to circumvent output-length constraints.

A structured human validation was conducted on a 25% double-blind stratified random sample... generating 168 cells for human review. This was justified by the impracticality of full manual review: Using a conservative estimate of ten minutes per cell... a single annotator performing an exhaustive review would require more than 110 hours of focused evaluation work. Three independent reviewers audited the sampled cells, consulting tool-specific capability summaries and the GitHub repositories directly where needed. Inter-rater agreement was measured using Fleiss' Kappa score, selected because the latter [Cohen's Kappa] is restricted to exactly two raters. The resulting score of 0.509 "indicates moderate inter-rater agreement, which is expected and acceptable in this context given the inherent interpretive difficulty of mapping loosely documented tool capabilities to formally defined taxonomy categories." Disagreements were resolved by majority vote, and the consensus was treated as ground truth for performance estimation.

The LLM-assisted pipeline achieved an accuracy of 84.5%, a precision of 78.4%, a recall of 72.7%, and an F1 score of 75.5% against the human consensus. The confusion matrix shows 40 true positives, 11 false positives, 15 false negatives, and 102 true negatives. The authors emphasize that Accuracy alone is therefore an unreliable indicator of pipeline performance in this setting, since it can be inflated by the prevalence of easy-to-identify negatives — of the 168 sampled cells, 32.7% were labeled Yes under human consensus, which means a naive classifier that predicted 'No capability' for every cell would achieve an accuracy of approximately 67.3%. The F1 score of 75.5% "indicates a statistically acceptable balance of precision and recall for this binary classification task, supporting the use of LLM-generated labels as a reliable approximation of ground truth for the remaining 75% of matrix cells."

The final Tool × Risk matrix reveals a pronounced skew: open-source tools provide dense coverage across technical security and operational process controls, but offer little direct support for governance oversight, legal and regulatory remedies, or financial and market-level interventions.

Within the technical security domain: "Model alignment and safety engineering (2.2 and 2.3) are supported by evaluation and red-teaming frameworks such as Promptfoo, DeepEval, PyRIT, Garak, NeMo Guardrails, and related suites, which collectively implement mechanisms such as rubric-based scoring, jailbreak and prompt-injection detection, adversarial attack libraries, and safety-oriented guardrails. Content safety controls (2.4) are similarly saturated... Infrastructure-oriented controls are somewhat less represented."

For operational process controls: "a diverse set of tools provides substantial support for testing and auditing (3.1), including automated red-teaming, regression testing, and batch evaluation pipelines, as well as for data governance (3.2), especially around PII detection, leakage testing, and privacy-preserving transformations. Post-deployment monitoring (3.5) is another area of relative strength. However, incident investigation (3.7) and staged deployment (3.4) receive only scattered coverage, often as emergent capabilities of tracing tools rather than as primary design goals. Access management (3.3) is similarly under-served."

In contrast, "governance oversight (1.x), transparency and accountability (4.x) beyond system documentation and risk disclosure, and downstream financial, legal, and market remedies (6.x–7.x) are largely unsupported by the tools examined. The authors explain: Governance, legal, and financial interventions are inherently institutional functions rooted in board-level decision-making, regulatory mandates, supervision, and compensation schemes which cannot be fully encoded as reusable software components."

Qualitatively, human disagreements "tended to cluster around derived mappings... and around categories that straddle technical and organizational domains. In particular, reviewers frequently diverged on whether observability-oriented capabilities should be counted as incident investigation or simply as post-deployment monitoring, and on whether limited documentation and reporting features warranted a positive mapping for transparency and risk disclosure."

The paper proposes a four-layer architecture to address the identified gaps:

1. Technical control layer: combines tools focused on alignment and safety engineering (taxonomy categories 2.2–2.3), content safety (2.4), and model-infrastructure protections (2.1). Representative tools include Promptfoo, Garak, PyRIT for pre-deployment evaluation and red teaming, NeMo Guardrails, LLM Guard, OpenGuardrails as middleware around LLM endpoints, and ModelScan, Adversarial Robustness Toolbox, Diffprivlib, PrivacyRaven for model and infrastructure scanning.

2. Observability and operations layer: supports testing and auditing (3.1), data-governance enforcement (3.2), post-deployment monitoring (3.5), and basic incident-response hooks (3.6), with Langfuse as tracing and logging spine, Arize Phoenix / OpenLIT for evaluation-aware monitoring, and scheduled regression testing tools.

3. Organizational governance layer: "encompasses board-level oversight, risk committees, safety decision frameworks, whistleblower protections, and transparency mechanisms (1.x, 4.x), which can consume evidence produced by the technical and observability layers... but are not encoded in the tools themselves."

4. Regulatory and market layer: comprises external standards, enforcement actions, compensation frameworks, and market-access decisions (6.x–7.x) that respond to incidents or systemic risk patterns rather than preventing them through code.

A financial services example is provided: "a technical control layer might combine Promptfoo, Garak, and PyRIT for pre-deployment red teaming of credit, fraud, and compliance use cases; NeMo Guardrails or LLM Guard to enforce runtime content and conduct policies... and ModelScan, Adversarial Robustness Toolbox, Diffprivlib, and PrivacyRaven to assess infrastructure, robustness, and privacy risks."

Future research directions include: (1) hands-on, scenario-based evaluation of tool stacks deploying "a red-teaming framework, a runtime guardrail engine, and an observability platform... to assess how well layered configurations actually mitigate concrete risks such as prompt injection, data leakage, hallucinations in retrieval-augmented generation, or policy non-compliance; (2) investigating the gap between documented and actual functionality by benchmarking multiple tools that claim to provide prompt-injection detection, PII masking, or hallucination scoring; (3) research on usability, integration, and organizational fit including configuration complexity, observability of its own behavior, interoperability with existing infrastructure... and the cognitive overhead it imposes on engineering and risk teams; and (4) extending the mapping to include commercial LLM governance platforms, cloud-provider offerings, and domain-specific solutions."

Key limitations acknowledged: the mapping is bounded by the scope and granularity of the MIT-derived taxonomy; the analysis is constrained by the state of the repositories at the time of ingestion (March 2026); the reliance on NotebookLM for initial extraction introduces model- and prompt-specific biases; and the study treats tools as largely independent units, whereas in practice they are deployed within complex socio-technical systems. The results should therefore be interpreted as a baseline map of verifiable, code-level capabilities rather than as an exhaustive account of all mitigation mechanisms present in real-world AI systems.

The study's main implications are twofold: "First, they underscore that no single tool and no purely technical tool stack, can provide end-to-end assurance... critical mitigation functions such as board-level oversight, conflict-of-interest management, whistleblower protections, regulatory compliance programs, and compensation or market-access decisions must be supplied by organizational and regulatory mechanisms that sit above the technical tooling. Second, the mapping makes it possible to reason about risk mitigation in terms of coverage and gaps rather than tool branding... organizations can ask which taxonomy categories are covered by their current stack, which are covered redundantly, and which remain exposed."

The paper concludes that operationalizing AI risk mitigation will require not only better tools, but also clearer mappings between these tools and the socio-technical systems in which they operate, and positions its taxonomy-driven analysis as a foundation for such work and a concrete starting point for enterprises seeking to move beyond ad-hoc experimentation toward deliberate, governance-aligned deployment of LLMs. Although the analysis focuses on open-source tools, the taxonomy-driven framework is designed to be applicable to proprietary solutions that organizations can incorporate into their workflows.

Based on the paper’s taxonomy-driven mapping methodology and findings, here are specific improvements for AI risk-mitigation and governance systems, along with the concrete capabilities the improved systems would have.

1. Replace closed, non-reproducible extraction with an open, configurable RAG pipeline.

The paper relies on NotebookLM, which limits reproducibility. An improved system should use an open-source retrieval-augmented generation stack with controllable chunking, embedding, retrieval, and model versions. This allows exact replication, auditing, and tuning of the pipeline.

2. Enforce a “zero-inference” rule during capability extraction.

The improved AI system must not infer a mitigation capability from marketing claims, tool names, or high-level documentation. It should require explicit code-level evidence, and when no evidence exists, explicitly record a true negative (“None Found”) rather than leaving the cell blank or guessing.

3. Automatically map tools to a structured risk taxonomy at sub-category granularity.

Instead of producing free-text descriptions, the system should output a standardized Tool × Risk matrix, e.g., 21 tools × 32 MIT-risk subcategories, with binary labels (Yes/No) for each cell.

4. Use a single-tool anchor and sequential category-wise extraction to avoid context-window overload.

Rather than processing all tools and all risk categories at once, the improved system should process one tool at a time, evaluate one taxonomy category after another, and use a strict auditor persona to keep outputs consistent and complete.

5. Require source-code citations for every positive mapping.

Every “Yes” cell must link to specific code artefacts, functions, or documentation sections in the repository. This makes the mapping auditable, reduces hallucination, and enables human reviewers to verify claims quickly.

6. Distinguish explicit from derived mappings.

The system should mark a mapping as “explicit” when the tool directly implements a capability, and “derived” when the capability reasonably follows from the tool’s behavior but is not named in taxonomy terms. Human reviewers can then focus their scrutiny on derived mappings, which were the main source of disagreement in the paper.

7. Automate human-in-the-loop validation with inter-rater agreement metrics.

The improved system should sample a stratified subset of matrix cells, route them to multiple independent reviewers, compute Fleiss’ Kappa, resolve disagreements by majority vote, and use the consensus to estimate precision, recall, and F1 on the unvalidated remainder. This gives trustworthy performance estimates without requiring a full manual audit.

8. Provide coverage and gap analysis across the risk taxonomy.

The system should aggregate the matrix and automatically show:

• which risk subcategories have dense coverage,

• which are partially covered,

• which are completely uncovered,

• and where redundant coverage indicates overlapping tools.

9. Recommend layered mitigation stacks based on the four-layer architecture.

The improved system should map tools into four layers — technical control, observability and operations, organizational governance, and regulatory/market — and propose a concrete stack for a given use case. For example, for a financial-services LLM deployment, it should recommend Promptfoo, Garak, and PyRIT for pre-deployment red-teaming; NeMo Guardrails or LLM Guard for runtime guardrails; ModelScan and Adversarial Robustness Toolkit for infrastructure scanning; and Langfuse plus Arize Phoenix for monitoring.

10. Expose unresolved organizational and regulatory risks.

The system should clearly flag risk subcategories that no software tool can fully address, such as board-level oversight, whistleblower protections, compensation frameworks, and market-access decisions. It should explain that these must be delivered by institutional mechanisms, not by code.

11. Support continuous feedback loops between technical monitoring and human oversight.

The improved system should connect post-deployment monitoring events (hallucination spikes, prompt-injection attempts, PII leaks) back to taxonomy categories, update the coverage matrix over time, and trigger alerts when a newly identified risk category is not addressed by the current tool stack.

12. Benchmark documented capability versus actual functionality.

The system should allow side-by-side testing of tools that claim the same capability, e.g., prompt-injection detection, PII masking, or hallucination scoring, and compare their documented mappings against real scenario-based evaluation results. This addresses the paper’s observed gap between documented and actual functionality.

13. Align with sector-specific regulatory frameworks.

The improved system should map both the MIT taxonomy and tool capabilities to sectoral frameworks such as the Financial Services AI Risk Management Framework, so that an enterprise can see which regulatory obligations are covered by its chosen open-source tools.

14. Extend mapping to commercial and proprietary tools.

Because the paper’s framework is taxonomy-driven, the improved system should use the same protocol to map commercial LLM governance platforms, cloud-provider offerings, and domain-specific risk tools — not just the 21 open-source repositories — so coverage comparison is possible across the whole vendor landscape.

• Ingest any AI tool’s GitHub repository, source code, and documentation, and automatically produce a citation-backed risk-capability matrix against a user-selected risk taxonomy.

• Run a fully reproducible LLM-assisted extraction pipeline that avoids hallucination by requiring explicit code evidence and by recording true negatives.

• Automatically identify coverage gaps in an organization’s current AI risk tooling, e.g., “You have strong coverage for content safety (2.4) and testing/auditing (3.1), but no coverage for incident investigation (3.7), access management (3.3), or governance oversight (1.x).”

• Recommend a concrete, layered tool stack for a given use case — e.g., pre-deployment red-teaming, runtime guardrails, observability, and privacy/robustness scanning — and explain which risk subcategories each tool combination addresses.

• Quantify the reliability of its own mappings using human-validated samples, Fleiss’ Kappa, precision, recall, and F1, so users know where to trust the mapping and where to review further.

• Route ambiguous “derived” mappings to human reviewers while auto-accepting explicit, code-cited mappings, making large-scale risk-mitigation audits practical.

• Continuously track tool updates by re-running the mapping when a repository changes, and flagging when a tool’s capability coverage shifts.

• Distinguish technical, organizational, and regulatory mitigation responsibilities, preventing enterprises from assuming that software alone can satisfy governance, legal, or board-level risk requirements.

• Support scenario-based validation, allowing teams to test whether a recommended tool stack actually mitigates concrete risks such as prompt injection, training-data leakage, hallucinated RAG outputs, or policy non-compliance.

• Create a common language between engineers and risk teams by translating technical tool capabilities into formal risk-taxonomy subcategories, thereby enabling governance-aligned AI deployment.

In short, the improved AI system becomes a taxonomy-driven, citation-backed, human-validated risk-mitigation mapping and recommendation engine — one that moves organizations from ad-hoc tool selection to deliberate, gap-aware, governance-aligned LLM deployment.

Rapid adoption of large language models (LLMs) in enterprise settings has introduced operational, security, and governance risks. As generative AI applications move from pilot to production, manual harm identification and mitigation are becoming difficult to scale. Although many tools support model evaluation, adversarial testing, runtime guardrails, and observability, the tooling landscape remains fragmented. Tools are typically designed for specific engineering tasks and described in technical terms that do not align with governance frameworks or risk taxonomies, making it difficult to determine which tools address which risks and where critical gaps remain. This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools. We map the capabilities of 21 prominent open-source tools to the 32 subcategories of the extended MIT AI Risk Mitigation and Response Taxonomy. An LLM-assisted retrieval-augmented generation pipeline analyzes source code and documentation to extract capabilities for each taxonomy category. Reliability assessment yielded moderate agreement (Fleiss' Kappa = 0.509) among three independent reviewers. The analysis reveals a highly skewed landscape in which tools cluster around technical and operational controls, while governance, legal and regulatory, and financial and market controls remain largely unaddressed. This motivates a layered risk-mitigation architecture combining tool-based controls with organizational and regulatory processes. The mapping protocol achieved an F1 score of 75.5% after majority voting. Overall, the study provides a practical mapping between enterprise AI risk categories and open-source mitigation capabilities, identifies where human oversight remains necessary, and presents a taxonomy-driven framework applicable to open-source and proprietary solutions.