QuantGuard: Learnable Rounding for Repairing Quantization-Conditioned Backdoors in LLMs
cs.CR
Submitted: 2026-06-28
Updated: 2026-08-27
Code: https://github.com/sdudaq/Quant_Guard
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Model quantization is a key technique for reducing storage and inference costs in large language model deployment.
Terminology
Abstract
Model quantization is a key technique for reducing storage and inference costs in large language model deployment. However, recent studies show that the discretization and rounding errors introduced by quantization can be exploited by adversaries to construct quantization-conditioned backdoor (QCB) attacks. Under such attacks, malicious behavior remains dormant at full precision and activates only after quantization, thereby bypassing conventional security auditing and detection. To address this threat, we propose QuantGuard, a proactive pre-quantization defense that learns safe rounding adjustments through differentiable optimization. Our method introduces differentiable rounding control variables and combines error-guided rounding reversal constraints, output-distribution consistency, and weight-distance regularization to regulate critical rounding behaviors. Crucially, QuantGuard utilizes only a small calibration dataset and does not modify existing quantization algorithms. This design disrupts the alignment between attacker-crafted weight patterns and quantization boundaries, suppressing post-quantization backdoor activation while preserving model functionality and performance. We conduct systematic experiments on six mainstream LLMs (including the LLaMA-3 and Qwen2.5-Coder) using three quantization precisions (INT8, FP4, and NF4) across three representative scenarios: vulnerable code generation, content injection, and over-refusal. The results show that QuantGuard consistently mitigates QCB attacks, reducing the attack success rate to a level comparable to the clean model while largely preserving general capability. With low computational overhead, QuantGuard provides a practical defense for secure quantized LLM deployment.
Sources
- Program Synthesis with Large Language Models
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Evaluating Large Language Models Trained on Code
- Rounding-Guided Backdoor Injection in Deep Learning Model Quantization
- SynGhost: Invisible and Universal Task-agnostic Backdoor Attack via Syntactic Transfer
- Mind the Gap: A Practical Attack on GGUF Quantization
- Extreme Compression of Large Language Models via Additive Quantization
- The Llama 3 Herd of Models
- DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
- Instruction Tuning for Secure Code Generation
- Measuring Massive Multitask Language Understanding
- Qwen2.5-Coder Technical Report
- StarCoder: may the source be with you!
- Expose Before You Defend: Unifying and Enhancing Backdoor Defenses via Exposed Models
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- DeepSeek-V3 Technical Report
- LLM-FP4: 4-Bit Floating-Point Quantized Transformers
- CodePurify: Defend Backdoor Attacks on Neural Code Models via Entropy-based Purification
- A White Paper on Neural Network Quantization
- Instruction Tuning with GPT-4
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills