Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework
Zhaoqi Wang, Daqing He, Zijian Zhang, Ye Liu, Jiamou Liu, Zhirui Zeng, Zhan Qin, Zhen Li, Xin Li, Hongwei Yao, Jincheng An, Yong Liu, Yi Li, Qi Sun, Xiulei Liu, Liehuang Zhu
cs.CR, cs.AI
Submitted: 2026-08-05
Journal ref: Proceedings of the ACM Web Conference 2026, pages 2661-2672, 2026
Code: https://github.com/Aquarids/scr-pub
License: http://creativecommons.org/licenses/by/4.0/
The gist: While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks.
Terminology
Abstract
While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions.
Sources
- GPT-4 Technical Report
- DeepSeek-V3 Technical Report
- A Survey of Large Language Models
- Poisoning Retrieval Corpora by Injecting Adversarial Passages
- PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
- Certifiably Robust RAG against Retrieval Corruption
- Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation
- Differentially Private Generative Adversarial Network
- DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Prompts
- Privacy-Preserving Machine Learning: Methods, Challenges and Directions
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- Qwen2 Technical Report
- Unsupervised Dense Information Retrieval with Contrastive Learning
- Clinical ModernBERT: An efficient and long context encoder for biomedical text
- FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
- Graph Attention Networks
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills