Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody

arXiv:2611.06173 · cs.CR, cs.LG · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody".

Elias: Here's the gist of the paper. Deploying an explainable intrusion detection model from a validated laboratory setting into an operational forensic workflow requires guarantees that laboratory benchmarks do not test, namely that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable<ref:2611.06173#pg3>.

Nadia: Pipeline Architecture The paper presents MODEXRE, a forensic pipeline extending a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability into a dual-mode architecture governed by a verifiable chain of custody and a frozen, hash-checked model<ref:2611.06173#pg3>.

Elias: This separation establishes Laboratory mode for training and Formal mode for inference exclusively on real evidence under full chain-of-custody logging<ref:2611.06173#pg4>.

Nadia: The backend is modularly organized into custody, ingestion, ocsf, features, models, pipeline, and report modules<ref:2611.06173#pg10>.

Title and authors: Nadia: So we’re looking at a paper called "Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody". It sounds like they're tackling the big problem of moving AI models from a lab to real investigations.

Elias: Exactly. The core idea is that you need guarantees when you take a model that worked in a controlled lab and apply it to actual evidence, because the test data in the lab just isn't the same as what you find at a crime scene.

Priya: That sounds like it touches on privacy and measurement issues, right? Because if we can't trust what the model sees from real evidence, then any conclusion we draw from that model is questionable.

Nadia: Right. And this paper describes a way to build this system, MODEXRE. They’re using something called a dual-mode architecture, which means it has a training mode and an operation mode for investigations.

Elias: That separation is key because it lets them train models on public or synthetic data in the lab, but when you get real evidence, it switches over to this formal mode where everything is locked down and verified.

Priya: So, if we think about what that means for us, it suggests a way to handle the messy reality of forensic data sources—like logs from different security tools—without losing track of what’s actually happening.

Nadia: Right. And they use something called OCSF normalization to get all that different evidence together into one common format so the model can understand it, even when the features don't match up perfectly with what they were originally trained on.

Elias: That addresses a known issue where academic models and real-world data just don't speak the same language, which is a huge hurdle for deploying this kind of AI in practice.

Priya: But does normalizing the features always mean you get a reliable result? I worry that if you smooth out all the differences too much, you might accidentally hide some important signal.

Nadia: That’s what they had to deal with because they found that simple normalization wasn't enough, so they actually had to make some design changes based on failures during development.

Elias: They mentioned four specific corrections that came out of those failures, including fixing a leak from a timestamp artifact that was messing up the model’s performance.

Title and authors: Priya: So they weren't just happy with their initial setup; they had to actively hunt for bugs in how the system handled real data, which shows how much uncertainty is involved in this work.

Nadia: And one of those fixes was about feature coverage—they realized that a typical alert from a tool like Suricata only had four features out of the seventy-eight features the original reference model expected.

Elias: That’s why they implemented a change to use just six features for real evidence, which fixed the problem where, for example, an SSH brute-force alert got misclassified as normal traffic.

Priya: So it sounds like they had to go back and re-evaluate the very features that define an attack based on what they saw in actual data, not just what was in the initial academic paper.

Nadia: Right. And one of the biggest wins, according to their testing, was causal temporal aggregation. This is how they look at time sequences to get a much better score.

Elias: That change really paid off; it raised that multiclass F1 macro from zero point three nine four up to zero point nine three two on their test set <ref:2611.06173#pg1,multiclass F1 macro from 0>.

Priya: That jump is significant, but I wonder if that higher number is truly reflective of a reliable detection, or if it's just a measure of how much the model learned the specific sequence pattern they were targeting.

Nadia: It’s about making crossflow behavioral patterns visible to the classifier so it can spot things like port scanning more effectively, which is what that aggregation helps with.

Elias: And on top of that, they focused heavily on explainability using SHAP values. This isn't just for us to see *what* the model decided; it’s about proving *why* it made that decision.

Priya: Explainability is crucial for admissibility, I guess. If a forensic expert has to testify in court, they can’t just say the AI flagged something; they need to show them exactly which pieces of evidence led to that flag.

Nadia: Exactly. The SHAP attributions are reported as being consistent with what we already know about network attacks, which builds trust in the system's output.

Elias: But this leads into the bigger picture: how does this architecture actually help someone who is just trying to use a security tool on their daily job?

Title and authors: Priya: It allows them to verify that the AI system’s output can be tied unambiguously to a specific, unaltered model artifact that was previously validated, which is a major step toward operationalizing it.

Nadia: So, by tying every inference in formal mode to a frozen and hash-checked model, they are creating these strong identity guarantees for the forensic process.

Elias: That separation between the training lab and the real evidence analysis is what makes this dual-mode architecture so useful for establishing those kinds of guarantees.

Priya: But we have to be careful about what they *don't* guarantee, though. They noted that their certified model hasn't been tested on a full-scale, real dataset from the companion project yet.

Nadia: That’s a fair limitation, because you can’t really prove something is robust if you haven't run it against the entire scope of what it might encounter in the field.

Elias: They also highlighted another practical constraint: evidence-source coverage is limited because the parsers only cover common core fields instead of trying to handle every single vendor-specific dialect.

Priya: So while they solved a lot of feature gaps, they are still bound by what the tools used to extract the data can actually provide, which means some specialized evidence might still be missed.

Nadia: And another thing they pointed out is that their chain of custody implementation is at an intermediate level; it’s a hash-chain without digital signatures for integrity checks.

Elias: That’s a design choice that balances security with the complexity of the forensic workflow, but it means you don't get the highest level of cryptographic proof for every single handover.

Priya: So, to wrap up this paper on "Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody," we see a very rigorous pipeline that addresses the gaps between theoretical model performance and messy real-world evidence.

Nadia: It provides a concrete framework for how security researchers can actually move validated AI models into an operational workflow while keeping the integrity of the evidence traceable.

Elias: It’s about making sure that when you use this system, you can trace every single decision back to a specific, certified model artifact and every piece of data used to make it.

Priya: It shows us that for forensic AI to be useful, we have to focus not just on the detection accuracy number, but on proving exactly how the model arrived at its conclusion using verifiable steps.

The paper's summary: Nadia: So we're looking at how this paper, "Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody," wraps up its whole setup.

Elias: Basically, it’s about taking that AI model we validated in the lab and making it trustworthy enough to actually use on a real case.

Nadia: Right. They built this dual-mode system, so you have a training mode where everything is flexible, and then a formal mode for investigation where the model is frozen and locked down with this chain of custody logging.

Elias: That separation is what they call operationalizing model identity guarantees; it means every time you run the AI on real evidence, you are tied to that specific, hash-verified artifact from the lab.

Nadia: It’s a big deal for admissibility because they aren't just saying "this AI found something"; they are reporting the chain of custody integrity and showing per-prediction explainability as measurable system behaviors.

Elias: Exactly. They use SHAP attributions that line up with what we already know about network attacks, which gives forensic experts a way to verify if the AI’s output makes sense in the real world.

Nadia: And they show some solid numbers too, like this certified model hitting an F1 macro of zero point six two one four on their test set after they fixed that initial bug.

Elias: That jump from what it was before shows how much fixing those coverage gaps and using better temporal aggregation really helps the multiclass performance.

Nadia: But they also laid out some hard limits, which is important for us to know as practitioners. They admit that the certified model hasn't been tested on a full-scale real dataset from their larger project yet.

Elias: And they point out that the evidence source coverage is limited because the tools they use can only read common fields instead of every single vendor-specific detail.

Nadia: Plus, their chain of custody is at an intermediate level; it’s a hash chain without digital signatures for those handovers.

Elias: So, while they solved the core problem of moving validated AI into a formal workflow, they also highlighted where the limitations still lie in real-world deployment.

Nadia: It shows that building the pipeline is one thing, but making it robust across every possible piece of evidence is another whole challenge.

Elias: And that’s where we look next: how does this system actually play against the threats we see in things like jailbreaking and adversarial attacks?

The paper's improvements: Tom: So we're looking at what they suggest we do next to actually make this system better than just being certified in a lab setting.

Nadia: The paper points out that one of the big fixes they made involved tackling those design failures directly by confronting them with real evidence rather than just adjusting the code in isolation.

Elias: Right. They discovered a hidden certification bug, which they fixed by doing regression tests against actual data, and that’s a huge lesson about testing rigorously.

Nadia: And another improvement is around feature selection; they realized that using just six features for real evidence was necessary because the original model was expecting seventy-eight features.

Elias: That addresses the coverage gap by focusing on what actually matters in the data, which prevents things like a brute force alert from being misclassified as normal traffic because it’s missing key indicators.

Nadia: They also mention causal temporal aggregation as a major upgrade; this is how they look at time sequences to see crossflow behavioral patterns, and it raised that multiclass F1 macro from zero point three nine four up to zero point nine three two.

Elias: That jump is significant because it means the AI isn't just looking at individual data points but understanding the sequence of events, like detecting a port scan.

Nadia: And on the explainability side, they emphasize that making those SHAP attributions consistent with domain knowledge is crucial so forensic practitioners can actually trust what’s coming out of the report.

Elias: They want to keep pushing that traceability; tying every decision back to a specific model artifact means we can really audit the entire detection process for legal requirements.

Nadia: So, they are suggesting that future work needs to focus on stress testing this formal mode against even more heterogeneous evidence types than just Suricata alerts and firewall logs.

Elias: And I think we should also look at how they handle the intermediate chain of custody level; maybe moving toward digital signatures would provide a stronger guarantee for the handover process.

Nadia: It’s about moving from a hash chain to something with more cryptographic proof, which is a big step for ensuring that evidence integrity isn't just assumed but mathematically proven.

Elias: Exactly. If we can strengthen those underlying proofs, we get closer to making this AI system truly bulletproof for high-stakes forensic work.

Conclusion: Nadia: So to wrap up, this paper on the "Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody" shows how you can take a lab-validated model and give it real operational power.

Elias: It really hammers home that for AI to be admissible in forensics, you have to prove exactly which artifact is being used at every single decision point.

Nadia: The implication is that we can start moving away from black-box systems toward something where the forensic trail isn't just a log, but an auditable link back to the exact model version used.

Elias: That level of rigor changes how we think about deployment; it moves it from a simple tool to a verifiable component in a legal process.

Priya: From my side, what this means for the data is that as long as the system accurately reports its chain of custody and explains *why* it flagged something, we can start trusting the output more than just a raw detection number.

Nadia: Right. So yeah, we've seen how they built this dual-mode architecture to separate training from real evidence analysis while keeping everything under strict control.

Elias: It’s a solid piece of engineering that addresses the trust problem in AI deployment by tying model identity directly into the inference process.

Priya: It’s a good foundation, but we still have those limitations they mentioned, like not testing it on the full real dataset yet, so we can't assume perfect performance across every scenario.

Nadia: True. And I think the next big question is whether this level of proof is scalable enough to handle the sheer variety of evidence sources we deal with in a real investigation.

Elias: Yeah, and that leads us nicely into how these kinds of forensic pipelines interact with things like adversarial attacks or jailbreaking tries.

José Luis Vela, Carmen Pellicer

CESTE, University Center, Zaragoza, Spain

cs.CR, cs.LG

Submitted: 2026-11-04

Updated: 2026-11-04

Comments: 25 pages, 8 figures, 5 tables. Companion paper to arXiv:2607.00763

Code: https://github.com/PFM-Ceste/modexre

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 86/100

The gist: Deploying an explainable intrusion detection model from a validated laboratory setting into an operational forensic workflow requires guarantees that laboratory benchmarks do not test…

Key concepts

Dual-Mode Architecture
This design separates the model's lifecycle into two distinct modes: Laboratory mode for training and tuning without custody tracking, and Formal mode for inference on real evidence. This separation ensures that only models trained under controlled conditions are used in official investigations where a verifiable chain of custody is mandatory.
OCSF Normalization
This process standardizes data from different sources, such as Suricata alerts and CEF firewall logs, into a common format. This resolves the gap between academic models and real evidence by ensuring that heterogeneous data can be correctly mapped to the model's features, improving detection accuracy.
Causal Temporal Aggregation
This technique analyzes events over time by looking at how they flow into each other, rather than just analyzing individual events in isolation. This method makes complex behavioral patterns visible to the classifier, significantly improving its ability to detect sequential attacks like port scanning.
SHAP Explainability
SHAP values are used to show exactly which parts of the input evidence contributed most strongly to a model's final prediction. This provides per-prediction explainability, allowing forensic experts to verify that the AI's output is traceable and consistent with known domain knowledge.

Summary

Deploying an explainable intrusion detection model from a validated laboratory setting into an operational forensic workflow requires guarantees that laboratory benchmarks do not test, namely that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable<ref:2611. p. 1

Where the paper says it

2 Abstract Deploying an explainable intrusion detection model from a validated laboratory setting into an operational forensic workflow requires guarantees that laboratory benchmarks do not test, namely that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable. This paper presents MODEXRE, a forensic pipeline that extends a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability into a dual-mode architecture, Laboratory and Formal, governed by a verifiable chain of custody and a frozen, hash-checked model.

06173#pg3>.

Pipeline Architecture

The paper presents MODEXRE, a forensic pipeline extending a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability into a dual-mode architecture governed by a verifiable chain of custody and a frozen, hash-checked model<ref:2611.06173#pg3>. This separation establishes Laboratory mode for training and Formal mode for inference exclusively on real evidence under full chain-of-custody logging<ref:2611. p. 2

Where the paper says it

It did not, however, address what happens once a certified model leaves the laboratory: whether the exact validated artifact can be guaranteed unchanged when applied to a case, whether real evidence sources (intrusion detection alerts, firewall logs, raw packet captures) can populate the feature vector the model expects, and whether attack patterns that only manifest across sequences of events, such as port scanning, can be detected at all under an event-by-event classification scheme. This paper addresses that gap through MODEXRE (MOdular Digital EXamination and Reporting Engine), a forensic pipeline built around a single architectural principle: strict separation between a Laboratory mode, where models are trained, tuned, and certified without chain-of-custody obligations, and a formal mode, where a frozen, hash-verified certified model performs inference exclusively on real evidence under full chain-of-custody logging.

06173#pg4>. The backend is modularly organized into custody, ingestion, ocsf, features, models, pipeline, and report modules<ref:2611. p. 8
Where the paper says it

Dual-mode architecture: Laboratory certification and Formal inference on a frozen certified model. At implementation level, this separation is reflected in the backend's modular organization: custody/ (chain of custody), ingestion/ (evidence ingestion), ocsf/ (OCSF normalization), features/ (feature

06173#pg10>.

Evidence Normalization and Feature Coverage

The system utilizes OCSF normalization to create a common representation across disparate evidence sources like Suricata alerts and CEF firewall logs<ref:2611.06173#pg6>. This addresses the feature-coverage gap between academic reference models and real evidence by allowing heterogeneous data to populate the feature vector<ref:2611. p. 2

Where the paper says it

It did not, however, address what happens once a certified model leaves the laboratory: whether the exact validated artifact can be guaranteed unchanged when applied to a case, whether real evidence sources (intrusion detection alerts, firewall logs, raw packet captures) can populate the feature vector the model expects, and whether attack patterns that only manifest across sequences of events, such as port scanning, can be detected at all under an event-by-event classification scheme. This paper addresses that gap through MODEXRE (MOdular Digital EXamination and Reporting Engine), a forensic pipeline built around a single architectural principle: strict separation between a Laboratory mode, where models are trained, tuned, and certified without chain-of-custody obligations, and a formal mode, where a frozen, hash-verified certified model performs inference exclusively on real evidence under full chain-of-custody logging.

06173#pg5>. The paper notes that a typical Suricata alert reliably populates only 4 of the 78 features in the academic reference model, causing a real SSH brute-force alert to be misclassified as normal traffic<ref:2611. p. 1
Where the paper says it

662, regression-tested); a coverage gap between the 78-feature reference model and the fields real evidence can populate caused a real SSH alert to be misclassified, resolved with a six-feature representation; causal temporal aggregation raised Kitsune multiclass F1- macro from 0. 932; and a real packet capture revealed that the certification pipeline had always trained a binary classifier, moving from classifying 100% of legitimate traffic as malicious to about 87% correct Normal classification after correction.

06173#pg5>.

Design Decisions and Corrections

Four design decisions were triggered by observed failures during development, each requiring a correction<ref:2611.06173#pg5>. These included correcting an information leak from a timestamp artifact that inflated Kitsune performance, resolving a coverage gap by using a six-feature representation for real evidence, implementing causal temporal aggregation to raise multiclass F1-macro from 0. p. 1

Where the paper says it

Four design decisions, each triggered by an observed failure rather than anticipated in advance, shaped the final architecture: an information leak from a timestamp artifact inflated Kitsune performance before correction (F1-macro 0. 662, regression-tested); a coverage gap between the 78-feature reference model and the fields real evidence can populate caused a real SSH alert to be misclassified, resolved with a six-feature representation; causal temporal aggregation raised Kitsune multiclass F1- macro from 0.

394 to 0.932, and discovering a hidden binary-only certification bug<ref:2611. p. 3
Where the paper says it

932 by making crossflow behavioral patterns visible to a per-event classifier. Empirical discovery and correction of a hidden certification defect: a binary-only training bug, invisible on every benchmark dataset used during development, exposed only by evaluating the system against a real packet capture before and after correction, the most direct empirical validation reported in this work.

06173#pg5>.

Empirical Validation and Explainability

The final certified model (v3) achieves an F1-macro of 0.6214 on the recertified 18-category model, reflecting the real multiclass task once the certification defect was corrected<ref:2611. p. 1

Where the paper says it

9538 on CICIDS2017-derived evidence, and the recertified 18-category model (v3) reaches F1-macro = 0. 6214, a figure that reflects the real multiclass task once the certification defect was corrected, with specialized port-scan detectors reaching F1-macro up to 0.

06173#pg3>. SHAP attributions are reported as consistent with domain knowledge, and forensic KPIs such as chain-of-custody verification and per-prediction explainability are measured outcomes<ref:2611. p. 1
Where the paper says it

6214, a figure that reflects the real multiclass task once the certification defect was corrected, with specialized port-scan detectors reaching F1-macro up to 0. SHAP attributions are consistent with domain knowledge, and forensic KPIs, chain-of-custody verification, model integrity hashing, and per-prediction explainability, are reported as measured outcomes.

06173#pg3>. A case study on port scanning showed that the certified v1 model achieved a binary detector F1-macro of 0. p. 3
Where the paper says it

Section 4 presents experimental evaluation of the four design decisions and the final certified model. Section 5 presents a port-scanning case study with SHAP-based explainability.

9997 using source and destination port features<ref:2611. p. 6
Where the paper says it

4 Temporal flow aggregation for sequence-dependent attack detection A model trained on the features of an individual flow can correctly identify whether that specific flow is anomalous, but attacks such as port scanning manifest precisely in the relationship between many short flows directed from the same origin toward many destinations or ports, a relationship invisible when each flow is evaluated in isolation. Flow-based scan-detection literature addresses this by fragmenting the continuous traffic stream into fixed-size temporal windows and computing, per window and per source address, derived attributes such as the number of distinct destinations addresses or ports contacted.

06173#pg6>.

Forensic Implications and Regulatory Alignment

The dual-mode architecture operationalizes model-identity guarantees by tying every Formal-mode inference to a specific, hash-verified, previously certified model<ref:2611.06173#pg10>. This separation allows certified models to be more powerful than case-specific ones and provides an auditable training process<ref:2611. p. 2

Where the paper says it

This separation operationalizes, as a structural property of the architecture rather than a downstream audit, the requirement identified across the forensic admissibility literature that an artificial intelligence system's output can be tied unambiguously to a specific, unaltered, previously validated model (Section 2. The separation also resolves a practical constraint: case evidence is scarce and private, so training certified models on public and synthetic data in Laboratory mode, with real evidence confined to Formal mode, allows more powerful models than any trained ad hoc on a single case, through a fully auditable training process.

06173#pg10>. The architecture addresses admissibility requirements by reporting chain-of-custody integrity, model identity verification, and per-prediction explainability as measured system behaviors<ref:2611. p. 1
Where the paper says it

SHAP attributions are consistent with domain knowledge, and forensic KPIs, chain-of-custody verification, model integrity hashing, and per-prediction explainability, are reported as measured outcomes. Keywords: digital forensic investigation; chain of custody; explainable artificial intelligence; multiclass intrusion classification; flow aggregation; OCSF; forensic admissibility

06173#pg3>.

Limitations

The paper notes limitations including the fact that the certified model has not yet been validated on the full-scale real dataset from the companion project<ref:2611.06173#pg3>. Furthermore, evidence-source coverage is bound because parsers cover only common core fields rather than vendor-proprietary dialects<ref:2611. p. 3

Where the paper says it

By decoupling training data from case evidence, it also enables more powerful, auditable models than case-specific training could provide. Multi-source evidence normalization via OCSF: a common representation across Suricata alerts, CEF firewall logs, and raw packet captures that expose, and then resolve the feature-coverage gap between academic reference models and real evidence.

06173#pg10>. The implemented chain of custody sits at an intermediate guaranteed level, being hash-chain without digital signatures<ref:2611. p. 6
Where the paper says it

5 Synthesis and research gap The reviewed literature generates well-developed guarantee frameworks for digital chain of custody, largely independent of intrusion-detection systems; OCSF solves the event-normalization problem but not the question of how a model trained on laboratory-level feature coverage should be adapted to unequal coverage across real evidence sources; legal literature on AI admissibility explicitly demands explainability and traceability but does not descend to the level of specific design decisions that satisfy that demand; and flow-aggregation literature solves the technical detection problem without integrating it into a complete forensic pipeline with end-to-end integrity guarantees. MODEXRE sits at the intersection of these four bodies of knowledge, and its contribution is not a novel technique within any single one, but their integration into a single operational architecture: an explainable, frozen, and versioned classification model; an OCSF-based normalization layer that absorbs source heterogeneity; a verifiable chain of custody accompanying every operation; and a temporal aggregation stage that makes sequence-dependent attack patterns visible.

06173#pg10>. Finally, the regulatory alignment is read as a literature-grounded design condition rather than an empirically obtained procedural guarantee<ref:2611. p. 8
Where the paper says it

This guarantee, identified in Section 2. 3 as a practically necessary condition for AI admissibility as forensic evidence, is thus a structural property of the architecture rather than a downstream check.

06173#pg3>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Dual-mode certified architecture enables a Laboratory mode for training and tuning without chain-of-custody obligations, while a Formal mode applies a frozen, hash-verified model exclusively to real evidence under full chain-of-custody logging. This separation operationalizes model identity guarantees identified as necessary but rarely implemented for forensic AI admissibility.

  2. OCSF normalization resolves feature coverage gaps across heterogeneous evidence by providing a common representation across Suricata alerts, CEF firewall logs, and raw packet captures that expose, and then resolve the feature-coverage gap between academic reference models and real evidence.

  3. Causal temporal flow aggregation raises multiclass F1-macro from 0.394 to 0.932 by making crossflow behavioral patterns visible to a per-event classifier, specifically enabling the detection of sequence-dependent attacks like port scanning.

  4. SHAP-based KPIs operationalize explainability and traceability for admissibility by providing per-prediction explainability that is consistent with domain knowledge, allowing forensic practitioners to verify that an AI system's output can be tied unambiguously to a specific, unaltered, previously validated model.

  5. The system can detect hidden certification defects by performing direct confrontation with real evidence, specifically exposing a bug where the pipeline had silently trained a binary classifier throughout development before correcting it via regression tests.

Abstract

Deploying an explainable intrusion detection model from a validated laboratory setting into a forensic workflow requires guarantees that benchmarks do not test: that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable. We present MODEXRE, a forensic pipeline that extends a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability (arXiv:2607.00763) into a dual-mode architecture: a Laboratory mode, where models are trained and certified on public and synthetic data, and a Formal mode, where a frozen, hash-verified model performs inference on case evidence under a SHA-256 hash-chained chain of custody. Suricata alerts, CEF firewall logs and packet captures are normalized through OCSF. Four design decisions, each triggered by an observed failure, shaped the architecture: a timestamp leak that inflated Kitsune performance (F1-macro 0.984, 0.662 after correction); a feature-coverage gap that made the 78-feature reference model misclassify a real SSH alert; causal temporal flow aggregation, which raised Kitsune multiclass F1-macro from 0.394 to 0.932; and a certification defect, exposed only by a real packet capture, whereby the pipeline had always trained a binary classifier. After correction, the same capture went from 100% of events flagged as malicious to 87% classified as Normal, with the scanning host identified as Reconnaissance. The default four-class certified model reaches F1-macro = 0.9538 on CICIDS2017-derived evidence; the recertified 18-category model reaches 0.6214, reflecting the real multiclass task. Chain-of-custody verification, model integrity hashing and per-prediction SHAP explanations are reported as measured outcomes.

Sources

Related papers