Rotated Robustness: A Training-Free Defense against Bit-Flip Attacks on Large Language Models
The episode discusses a paper titled "Rotated Robustness: A Training-Free Defense against Bit-Flip Attacks on Large Language Models." Researchers found that single bit-flips in quantized weights can cause catastrophic model collapse due to outlier features. The proposed solution, Rotated Robustness (RoR), uses Householder transformations and a compact WY representation to spread outlier energy, offering strong protection against single-point fault attacks with minimal latency.