LoRA as Oracle
cs.CR, cs.AI
Submitted: 2026-01-16
Updated: 2026-08-27
License: http://creativecommons.org/licenses/by/4.0/
The gist: Practitioners increasingly deploy neural networks they did not train, and must audit them after the fact for hidden backdoors, without the training pipeline, the poisoned data, or knowledge of any
Terminology
Abstract
Practitioners increasingly deploy neural networks they did not train, and must audit them after the fact for hidden backdoors, without the training pipeline, the poisoned data, or knowledge of any trigger. We introduce a low-rank auditing lens built on a single observation: what a model has internalized and how it behaves are distinct axes that can diverge. Fitting a small low-rank adapter toward a hypothesis and reading the geometry of the resulting update, its energy relative to, and its alignment with, the frozen weights, measures internalization directly, independently of the model's output behavior. The lens earns its value where the two axes diverge: a backdoor is malicious internalization, a learned trigger-to-target shortcut, that behavioral auditing can miss. Reading it, we identify the backdoor's target class label-free and without any triggered data when the backdoor leaves a legible internalization signature, then erase the identified shortcut within the same low-rank subspace, uniquely coupling detection confidence to the size of the repair. Across four datasets and four architectures, our lens is the most consistent target identifier among defenses in its threat model; its rank-r repair removes backdoors while preserving clean accuracy where full-model baselines collapse it, at orders-of-magnitude lower parameter and memory cost; and it is the only method that both erases the backdoor and leaves a benign model intact, because it can decline to act when uncertain.
Sources
- SoK: The Last Line of Defense: On Backdoor Defense Evaluation
- Prototype-Guided Robust Learning against Backdoor Attacks
- Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
- FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- Label-Consistent Backdoor Attacks
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Vision Transformer for Small-Size Datasets
- LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
- LoBAM: LoRA-Based Backdoor Attack on Model Merging
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs