Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models

arXiv:2602.12183 · cs.CR, cs.SE · Submitted 2026-02-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models".

Elias: Here's the gist of the paper. The proposed approach introduces SiamXBERT, a robust and data-efficient Siamese meta-learning framework empowered by a transformer-based language model for unknown attack detection in IoT networks A dual-modality feature representation is constructed by integrating flow-level and packet-level information, enabling richer behavioral modeling while remaining compatible with encrypted traffic.

Nadia: Through meta-learning, the model rapidly adapts to new attack types using only a small number of labeled samples and generalizes to previously unseen behaviors.

Elias: Extensive experiments on representative IoT intrusion datasets demonstrate that SiamXBERT consistently outperforms state-of-the-art baselines under both within-dataset and cross-dataset settings while requiring significantly less training data, achieving up to 78.8% improvement in unknown F1-score.

Nadia: Pruning redundant reasoning tokens The core motivation for the work stems from the limitations of existing methods that rely on large labeled datasets or raw payload inspection.

Title and authors: Nadia: So, we're looking at the paper titled "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models," and it sounds like they are tackling a big problem with zero-day threats where you don't have lots of labeled data.

Elias: Right. The title points to using language models for unknown attack detection in IoT, which is significant because those things evolve so fast that traditional methods struggle when they see something new.

Nadia: Exactly, and the authors are Shan Ali, Feifei Niu, Paria Shirani, Lionel C. Briand, and they propose SiamXBERT as their solution to this data scarcity issue.

Elias: SiamXBERT is their Siamese meta-learning framework that uses a transformer-based language model to detect unknown attacks in IoT networks without needing massive labeled datasets.

Priya: I'm curious how they plan on proving that it's efficient when you are dealing with real network traffic, not just clean lab data.

Nadia: Well, the core idea is building a dual-modality feature representation by combining flow-level and packet-level information to model behavior better while still being compatible with encrypted traffic.

Elias: That dual approach means they're not relying on raw payloads, which is smart because that data is often inaccessible in real IoT environments.

Nadia: They claim this allows the model to rapidly adapt to new attack types using only a small number of labeled samples and then generalize well to behaviors it hasn't seen before.

Elias: That sounds like meta-learning allowing for fast adaptation, which is usually a good sign when you're dealing with unseen threats.

The paper's summary: Nadia: So, what the authors summarize in the "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models" paper is that existing methods often fail because they need huge datasets or they look at raw payloads which are encrypted.

Elias: They summarize that SiamXBERT solves this by using a domain-specific SecBERT model fine-tuned with only one hundred samples per class, which provides extreme data efficiency while keeping the defense payload-free in cross-dataset scenarios <ref:2602.12183#pg3,with only 100 samples per class>.

Priya: So, the summary highlights this efficiency as a key feature, but I need to know what that actually means for the accuracy when we test it against real-world shifts.

Nadia: The paper summarizes their methodology by constructing a dual-modality representation by integrating flow and packet information through five stages: extraction, integration, preprocessing, meta-training with threshold selection, and finally testing.

Elias: Specifically for the meta-training stage, they decompose training into binary meta-tasks where each task is one per attack type using a triplet loss function to compute similarity between anchor-positive and anchor-negative embedding pairs.

Nadia: That triplet loss is what drives the learning process, allowing it to learn how attacks are similar even if they look different from previous ones in the training set.

Priya: I wonder about that threshold selection part; how do they know when the model has learned enough to stop guessing and start detecting?

Elias: The meta-training stage specifically involves threshold selection, where they dynamically calibrate the detection threshold based on simulated unknown scenarios to maximize the unknown class F1-score for a pseudo-unknown class Ci.

The paper's improvements: Nadia: Now let's talk about the specific improvements suggested by this work in "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models." They focus heavily on overcoming the three limitations they identified earlier.

Elias: They suggest fine-tuning a domainspecific SecBERT model with only one hundred samples per class, which is a major improvement over the hundreds to tens of thousands of samples that most existing methods require for stable performance <ref:2602.12183#pg3,fine-tuning a domainspecific SecBERT model with only 100 samples per class>.

Priya: So, if we take that down to just one hundred samples per class, does the resulting detection still hold up when we move from one dataset to another <ref:2602.12183#pg3>?

Nadia: They claim this approach maintains payload-free resilience in cross-dataset scenarios and demonstrates superiority by conducting explicit cross-dataset evaluations.

Elias: They also emphasize the benefit of fusing flow features from Zeek with packet features from DPKT, which results in a unified representation with seventy features that captures both global behavioral patterns and fine-grained local dynamics.

Priya: Those seventy features sound comprehensive, but I'm concerned about feature importance selection; if you select the wrong ones, does the whole system fail?

Nadia: They found that feature importance selection has a pronounced positive effect across nearly all configurations, resulting in clear improvements in unknown F1-score on both datasets.

Elias: Plus, they note that metric learning is important because SiamXBERT aligns naturally with the open-set nature of unknown attack detection by learning a similarity space instead of enforcing closed-set class boundaries.

Conclusion: Nadia: So, wrapping up the "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models" paper, they show that SiamXBERT consistently outperforms state-of-the-art baselines for unknown attack detection in both within and cross-dataset settings.

Elias: The key finding is that LLM-based models achieve competitive performance in small settings but consistently outperform other machine learning and deep learning baselines when dealing with large scale IoT attacks.

Priya: I'm interested in what this means for practical deployment; if it’s so robust, how much data do we actually need to train this system before we can deploy it reliably?

Nadia: They claim SiamXBERT achieves up to seventy-eight point eight percent improvement in the unknown F1-score while requiring significantly less training data than previous methods.

Elias: The asymmetry they observed between cross- and within-dataset performance comes down to differences in dataset characteristics and calibration effects, suggesting it learns a broader representation on one set that absorbs samples from another.

Priya: From my side, the limitation they state is that their evaluation is restricted to intra-dataset holdouts, which means we need more validation on truly unseen distributions before trusting the cross-dataset claims completely.

Nadia: Exactly, so while it's very effective with less data and payload-free features like flow and packet information, we still need to watch how it performs when the attack landscape changes drastically in a new environment.

Elias: So, in summary, SiamXBERT offers a way to use language models for unknown attacks that is much more data-efficient than what we've seen before.

Nadia: That was our discussion on "Data-Efficient Unknown Attack Detection in IoT Networks Using Pretrained Language Models."

Elias: We’re ready to look at the next paper when we come back.

University of Ottawa · Research Ireland Lero Centre

cs.CR, cs.SE

Submitted: 2026-02-12

Updated: 2026-11-11

Comments: 19 pages, 2 figures

Code: https://github.com/jackaduma/SecBERT

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 80/100

The gist: The proposed approach introduces SiamXBERT, a robust and data-efficient Siamese meta-learning framework empowered by a transformer-based language model for unknown attack detection in IoT networks A…

Key concepts

SiamXBERT
A robust Siamese meta-learning framework that uses a transformer language model for unknown attack detection. It learns how to detect new attacks quickly by adapting from very few labeled samples, making it highly data-efficient.
Dual-Modality Feature Representation
This involves combining two types of information: flow-level features (like traffic patterns over time) and packet-level features (details of individual packets). This dual approach creates a richer understanding of network behavior while remaining compatible with encrypted traffic.
Meta-Training with Threshold Selection
The model is trained by breaking down the task into many small binary meta-tasks. During this process, the system learns to dynamically select the best detection threshold for unknown attacks based on simulated scenarios, optimizing performance for unseen threats.

Terminology

Summary

The proposed approach introduces SiamXBERT, a robust and data-efficient Siamese meta-learning framework empowered by a transformer-based language model for unknown attack detection in IoT networks A dual-modality feature representation is constructed by integrating flow-level and packet-level information, enabling richer behavioral modeling while remaining compatible with encrypted traffic. p. 0

Where the paper says it

To address these limitations, we propose SiamXBERT, a robust and data-efficient Siamese meta-learning framework empowered by a transformerbased language model for unknown attack detection. The proposed approach constructs a dual-modality feature representation by integrating flow-level and packet-level information, enabling richer behavioral modeling while remaining compatible with encrypted traffic.

Through meta-learning, the model rapidly adapts to new attack types using only a small number of labeled samples and generalizes to previously unseen behaviors. p. 0
Where the paper says it

The proposed approach constructs a dual-modality feature representation by integrating flow-level and packet-level information, enabling richer behavioral modeling while remaining compatible with encrypted traffic. Through meta-learning, the model rapidly adapts to new attack types using only a small number of labeled samples and generalizes to previously unseen behaviors.

Extensive experiments on representative IoT intrusion datasets demonstrate that SiamXBERT consistently outperforms state-of-the-art baselines under both within-dataset and cross-dataset settings while requiring significantly less training data, achieving up to 78. p. 0
Where the paper says it

Through meta-learning, the model rapidly adapts to new attack types using only a small number of labeled samples and generalizes to previously unseen behaviors. Extensive experiments on representative IoT intrusion datasets demonstrate that SiamXBERT consistently outperforms stateof-the-art baselines under both within-dataset and cross-dataset settings while requiring significantly less training data, achieving up to 78.

8% improvement in unknown F1-score. p. 0
Where the paper says it

Extensive experiments on representative IoT intrusion datasets demonstrate that SiamXBERT consistently outperforms stateof-the-art baselines under both within-dataset and cross-dataset settings while requiring significantly less training data, achieving up to 78. 8% improvement in unknown F1-score.

Pruning redundant reasoning tokens

The core motivation for the work stems from the limitations of existing methods that rely on large labeled datasets or raw payload inspection. Specifically, existing solutions often struggle to identify unknown attacks because they are optimized under closed-set or intra-dataset settings, limiting their generalization to unseen behaviors. p. 0

Where the paper says it

(ii) Many methods rely on raw payload inspection [23, 28, 29], which become ineffective in encrypted IoT traffic and raise privacy concerns. (iii) Detectors are commonly optimized under closed-set or intra-dataset settings and therefore exhibit limited generalization to unseen behaviors or cross-dataset distribution shifts [23, 30, 31].

The authors address these gaps by fine-tuning a domain-specific SecBERT model with only 100 samples per class, providing extreme data efficiency while maintaining payload-free resilience in cross-dataset scenarios. p. 2
Where the paper says it

Our work addresses these gaps by fine-tuning a domainspecific SecBERT model with only 100 samples per class (i. , 500 and 2500 samples, respectively), providing extreme data efficiency while maintaining payload-free resilience in cross-dataset scenarios.

How the training signal is built

SiamXBERT constructs a dual-modality feature representation by integrating flow-level and packet-level information. This process involves five main stages: (i) feature extraction, (ii) dual-modality feature integration, (iii) feature preprocessing and selection, (iv) meta-training with threshold selection, and (v) meta-testing for known and unknown attack detection. p. 2

Where the paper says it

SiamXBERT captures both flow-level and packet-level features from PCAP files to provide a holistic representation of network behavior. As illustrated in Figures 1 and 2, SiamXBERT consists of five main stages: (i) feature extraction, (ii) dual-modality feature integration, (iii) feature preprocessing and selection, (iv) meta-training with threshold selection, and (v) meta-testing for known and unknown attack detection.

In the meta-training stage, the process decomposes training into a collection of binary meta-tasks (MT), one per attack type: MTi = (B, Ci), Ci ∈ C. p. 4
Where the paper says it

, Cn}, and the unknown class by U. As shown in the pipeline in Figure 2, we first decompose the training process into a collection of binary meta-tasks (MT), one per attack type: MTi = (B, Ci), Ci ∈ C.

For each task MTi, the model is fine-tuned using a triplet loss function that computes similarity between anchor-positive and anchor-negative embedding pairs. p. 4
Where the paper says it

As depicted in Figure 2, SecBERT encodes each input prompt (xa, xp, xn) to a fixed-dimensional embedding, denoted as fθ(x), which is used as the sequencelevel representation for similarity learning. The network is then optimized using a triplet loss function, which computes the similarity between the anchor-positive and anchor-negative embedding pairs, aiming to pull fθ(xa) and fθ(xp) closer while pushing fθ(xa) and fθ(xn) apart.

Where the gains come from

The performance of SiamXBERT is driven by several key design choices that address the challenges in IoT security. The study systematically benchmarks SiamXBERT against four state-of-the-art (SOTA) baselines for unknown attack detection under both within- and cross-dataset scenarios. p. 1

Where the paper says it

By leveraging multi-level feature extraction and meta-learning, SiamXBERT accurately distinguishes novel attack signatures from legitimate network traffic, even in low-data regimes. • We systematically benchmark SiamXBERT against four state-of-the-art (SOTA) baselines for unknown attack detection under both within- and cross-dataset scenarios.

Key findings include:

  1. LLM-based models achieve competitive performance in small-scale settings and consistently outperform ML and DL baselines in large-scale IoT attack detection. p. 7

    Where the paper says it

    The relatively small performance gap among BERT, SecureBERT+, and SecBERT further suggests that both largescale pretraining and domain adaptation contribute to effective representation learning in this regime. Answer to RQ1: LLM-based models achieve competitive performance in small-scale settings and consistently outperform ML and DL baselines in large-scale IoT attack detection.

  2. The choice of feature sets has a clear impact on the performance of unknown attack detection, with combining flow-level and packet-level features yielding consistent improvements in U-F1, particularly on CICIoT2023. p. 7

    Where the paper says it

    Combining flow-level and packet-level features yields consistent improvements in U-F1, particularly on CICIoT2023, where the Flow+Packet configuration improves U-F1 by approximately 4-6% over the best single-modality setting. Impact of derived features: Augmenting feature sets with derived features generally increases unknown recall (often exceeding 10 percentage points), yet fails to yield consistent UF1 improvements.

  3. Feature importance selection has a pronounced positive effect across nearly all configurations, resulting in clear improvements in U-F1 on both datasets. p. 8

    Where the paper says it

    9 Impact of feature selection: Feature selection has a pronounced positive effect across nearly all configurations, resulting in clear improvements in U-F1 on both datasets. In particular, it improves U-F1 by over 30% on IoT-23 (e.

  4. Metric learning matters because SiamXBERT aligns naturally with the open-set nature of unknown attack detection by learning a similarity space instead of enforcing closed-set class boundaries. p. 0

    Where the paper says it

    To overcome these challenges, we introduce SiamXBERT, a robust and data-efficient approach for detecting unknown attacks. Instead of learning closed-set decision boundaries, SiamXBERT adopts a meta-learning strategy modeling similarity relationships between benign and attack behaviors.

Cross-dataset robustness and generalization

SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry observed between cross- and within-dataset performance suggests that models trained on CICIoT2023 tend to learn a broader, more complex representation of known attacks, thereby causing many IoT-23 attack samples to be absorbed into the known attack manifold. p. 9

Where the paper says it

This asymmetry suggests that models trained on CICIoT2023 tend to learn a broader, more complex representation of known attacks, thereby causing many IoT-23 attack samples to be absorbed into the known attack manifold, thereby reducing the effectiveness of unknown attack detection. The observed asymmetry between cross-dataset directions TABLE V CROSS-DATASET UNKNOWN ATTACK DETECTION RESULTS (RQ4).

Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

The results show that SiamXBERT consistently outperforms SOTA baselines for unknown attack detection, on both datasets, in a within-dataset setting, achieving higher unknown F1-scores despite being trained with significantly fewer labeled samples. p. 8
Where the paper says it

264 on C&C-Mirai), indicating greater robustness in detecting unknown attacks. Answer to RQ3: The results show that SiamXBERT consistently outperforms SOTA baselines for unknown attack detection, on both datasets, in a within-dataset setting, achieving higher unknown F1-scores despite being trained with significantly fewer labeled samples.

The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

p. 9
Where the paper says it

086 SiamXBERT⋆ 0. 192 stems from differences in dataset characteristics and calibration effects.

The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

p. 9
Where the paper says it

Also, decision thresholds and similarity calibration are derived from the training phase, leading to distributional mismatch in cross-dataset testing and amplifying these effects. Answer to RQ4: Overall, SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset.

The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects. The paper concludes that SiamXBERT demonstrates strong cross-dataset robustness compared to SOTA methods, particularly when trained on IoT-23 and tested on CICIoT2023 dataset. The asymmetry between cross-dataset directions stems from differences in dataset characteristics and calibration effects.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Use SiamXBERT to detect unknown attacks using only 100 samples per class, achieving extreme data efficiency while maintaining payload-free resilience in cross-dataset scenarios. This allows for robust defense against zero-day threats without requiring massive labeled datasets.

  2. Implement the dual-modality feature integration by combining Zeek's flow features and DPKT's packet features, which results in a unified representation with 70 features. This enables the model to capture both global behavioral patterns and fine-grained local dynamics.

  3. Utilize derived features like byte ratio, origin packet rate, origin byte rate, and direction to explicitly capture temporal intensity and directional asymmetry—properties of network flows. This helps distinguish between attacks with similar aggregate volumes but different directional characteristics.

  4. Employ the meta-training with threshold selection process (Algorithm 1) to dynamically calibrate the detection threshold based on simulated unknown scenarios. This ensures that the final decision threshold is optimized to maximize the unknown class F1-score for pseudo-unknown class Ci.

  5. Perform inference using Algorithm 2, which aggregates similarity scores from all trained meta-models and assigns a query to U if s is below the predetermined threshold tau star, enabling real-time detection of unseen attacks.

Abstract

The rapid evolution of cyberattacks continues to drive the emergence of unknown (zero-day) threats, posing significant challenges for network intrusion detection systems in Internet of Things (IoT) networks. Existing machine learning and deep learning approaches typically rely on large labeled datasets, payload inspection, or closed-set classification, limiting their effectiveness under data scarcity, encrypted traffic, and distribution shifts. To address these limitations, we propose SiamXBERT, a data-efficient, open-set Siamese meta-learning framework built on a pretrained language model for unknown-attack detection. The framework constructs a dual-modality feature representation by integrating flow-level and packet-level information, enabling richer behavioral modeling while remaining compatible with encrypted traffic. The model learns similarity patterns for known benign and attack classes using 100 labeled samples per class. It flags traffic that does not sufficiently match any known class as potentially unknown, using a threshold set without unknown-attack data. We evaluate SiamXBERT on two IoT intrusion detection datasets against five state-of-the-art detection methods. When training and testing are performed on the same dataset at this labeling budget, SiamXBERT attains the highest unknown F1-score, exceeding the strongest competing method on each dataset by 84.8% and 37.4%; the improvement is statistically significant against all five methods on one dataset and against four of the five on the other. When a model trained on one dataset is tested on another, SiamXBERT transfers effectively in one direction but degrades in the reverse, reflecting the rejection behavior learned from the source traffic rather than the choice of threshold. These results highlight the practicality of SiamXBERT for data-efficient unknown attack detection in real-world IoT environments.

Sources

Related papers