General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference

arXiv:2606.25349 · cs.CR · Submitted 2026-06-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference".

Elias: Here's the gist of the paper. In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications, where key switching primarily occurs and dominates computational overhead in both FHE-based and hybrid FHE-MPC systems<ref:2606.25349#pg2> Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability<ref:2606.25349#pg2>.

Nadia: Pruning redundant reasoning tokens The authors propose a preprocessing-assisted method for secure attention computation that reduces online key-switching without modifying existing packing strategies<ref:2606.25349#pg2> This method reorganizes ciphertext attention computation through preprocessing so that part of the ciphertext transformation workload is shifted from the online phase to preprocessing<ref:2606.25349#pg2>.

Elias: Furthermore, they introduce a fused key-switch technique targeting the multiplication-followed-by-rotation pattern in RNS-CKKS matrix multiplication, which combines relinearization and rotation into a single procedure to reduce associated computation costs<ref:2606.25349#pg2>.

Nadia: How the training signal is built The proposed preprocessing method decomposes attention into precomputable operations and online interactions to reduce online inference-phase key switching without modifying existing packing strategies<ref:2606.25349#pg2>.

Title and authors: Nadia: So this paper is called General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference. It sounds really technical, but basically, it’s about making those large linear layers in secure AI models work better without having to change how we pack the data.

Elias: Right. The authors are Wenshao Yang and Zhenhua Liu and Dongdong Yao. They focus on this problem where we use Fully Homomorphic Encryption, or FHE, for these Transformer layers, and key switching is the main thing slowing everything down.

Nadia: It seems like they're tackling a common bottleneck in secure AI inference—that constant key switching that just keeps adding massive computational cost to the process.

Elias: Exactly. They’re targeting this overhead from a packing-independent perspective, which means they aren't tied to one specific way we arrange our data, which is pretty important since existing optimizations are often just packing-specific and don't work everywhere.

Priya: From my side, I’m thinking about what this means for the actual measurement process. If you can reduce the online key switching without changing how the plaintext is packed, that makes deployment much more flexible for different hardware setups.

Nadia: That’s a good point about flexibility. It suggests we might be able to use these secure Transformer models on more varied resource constraints than before.

The paper's summary: Elias: So, the core idea of the General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference is proposing a preprocessing-assisted method for computing attention. They decompose the attention calculation into parts that you can do beforehand and parts that happen online.

Nadia: It sounds like they are trying to shift some of the heavy lifting—the key switching—from the real-time inference phase into a preparation phase, which should help keep things running faster during actual use.

Priya: But then they introduce this storage trade-off idea because moving that work offline means you need more storage for those precomputed ciphertexts, so they have to figure out how to manage that memory versus the communication overhead online.

Elias: That’s right. They propose these storage-communication trade-off techniques that replace those huge precomputed ciphertexts with just a small amount of online communication instead. This lets you keep the benefit of preprocessing while keeping the system deployable even when storage is tight, and it works for both FHE and hybrid MPC systems.

Nadia: So, they’re giving us a way to handle the key switching problem by splitting it up—some offline prep, some online interaction—and managing the resulting memory trade-offs smartly.

The paper's improvements: Nadia: They have three main improvements they lay out. First, that packing-independent preprocessing method for attention computation that cuts down on online key switching without needing to touch any of our current packing strategies.

Elias: And second, they introduce a fused key-switching technique specifically for the multiplication followed by rotation pattern in RNS CKKS matrix multiplication. They combine relinearization and rotation into one single procedure to cut down the computational cost of that step.

Priya: That sounds like a nice win for the raw computation speed, but I wonder how much data you’re actually moving around during that fused operation compared to doing them separately?

Elias: Well, they say it eliminates one ModDown stage in RNS CKKS matrix multiplication compared to the conventional approaches, which directly reduces the associated computation cost. That's a concrete reduction in the operations happening on the hardware.

Nadia: So we get a faster multiplication step because of that fused technique, and we still get that packing-independent attention method for handling key switching. It’s like they’re attacking two different parts of the problem at once, which is smart.

Conclusion: Elias: So to wrap up the General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference, we have a preprocessing-assisted method that handles attention key switching without touching packing, a fused technique that cuts down on multiplication cost by combining relinearization and rotation, and trade-off techniques for storage versus communication.

Nadia: This means we can potentially deploy these secure Transformer models in ways we couldn't before because the latency-critical cryptographic parts are handled more efficiently. It keeps compatibility with our existing systems while offering a path to lower operational costs.

Priya: I just want to say that the storage-communication trade-off is key; it shows that you don't have to choose one perfect solution, but you can balance memory usage against how much data you send back online.

Elias: Agreed. The authors confirm that these protocols securely compute the desired outputs under the standard semi-honest model and the INDCPA assumption of RLWE, which gives us some confidence in the security guarantees for this method.

Nadia: So it’s a mechanism that balances computation, storage, and communication costs while keeping online secure Transformer inference fast enough for real use. That’s what they proved with this paper on General Linear-Layer Algorithms for Privacy-Preserving Transformer Inference.

Wenshao Yang, Zhenhua Liu, Dongdong Yao

cs.CR

Submitted: 2026-06-24

Updated: 2026-11-01

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 81/100

The gist: In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications…

Key concepts

Key Switching
This is a computationally expensive operation in secure inference where the encryption keys must be updated during computation. The paper focuses on minimizing this cost, which is often the main bottleneck in running Transformer models securely using FHE.
Preprocessing-Assisted Attention Computation
Instead of performing all key-switching during the live inference phase, this method reorganizes attention calculations. It moves part of the transformation workload to a preprocessing step, making it compatible with various plaintext packing strategies and reducing online switching overhead.
Fused Key-Switching Technique
This optimization combines two separate cryptographic steps—relinearization and rotation—into one single procedure when multiplying matrices in RNS-CKKS. This reduces the total number of computational stages required for this specific matrix operation, lowering the overall cost.
Storage-Communication Trade-off
To manage memory constraints, this strategy replaces storing large precomputed ciphertexts with sending smaller amounts of plaintext data during online communication. This allows systems to operate effectively under different resource limits by trading storage space for faster online transmission.

Terminology

Summary

In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications, where key switching primarily occurs and dominates computational overhead in both FHE-based and hybrid FHE-MPC systems<ref:2606. p. 025349#pg2> Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability<ref:2606. p. 025349#pg2>.

Pruning redundant reasoning tokens

The authors propose a preprocessing-assisted method for secure attention computation that reduces online key-switching without modifying existing packing strategies<ref:2606.25349#pg2> This method reorganizes ciphertext attention computation through preprocessing so that part of the ciphertext transformation workload is shifted from the online phase to preprocessing<ref:2606. p. 025349#pg2>. Furthermore, they introduce a fused key-switch technique targeting the multiplication-followed-by-rotation pattern in RNS-CKKS matrix multiplication, which combines relinearization and rotation into a single procedure to reduce associated computation costs<ref:2606. p. 025349#pg2>.

How the training signal is built

The proposed preprocessing method decomposes attention into precomputable operations and online interactions to reduce online inference-phase key switching without modifying existing packing strategies<ref:2606.25349#pg2>. To address additional storage requirements from shifting key switching offline, they propose storage-communication trade-off techniques that replace large precomputed ciphertexts with modest online communication<ref:2606. p. 025349#pg2>. These techniques enable flexible deployment under varying resource constraints<ref:2606. p. 025349#pg2>.

Where the gains come from

The core idea of the preprocessing-assisted method is to shift part of the ciphertext transformation workload from the online phase to preprocessing, making it compatible with a range of plaintext packing strategies<ref:2606.25349#pg2>. The attention score matrix is decomposed into several ciphertext multiplications and offline precomputable ciphertext transformations, which moves a significant portion of the costly key-switching operations into the preprocessing stage<ref:2606. p. 325349#pg2>. The fused key-switching technique reduces the cost of multiplication followed by rotation by combining relinearization and rotation into one procedure, eliminating one ModDown stage compared to conventional approaches<ref:2606. p. 125349#pg2>.

Mitigating overhead via trade-offs

The storage–communication trade-off strategy replaces large transformed ciphertext storage with a small amount of online plaintext communication, preserving the benefit of preprocessing while lowering storage costs<ref:2606.25349#pg2>. This strategy is adapted for both FHE-based and hybrid MPC–FHE Transformer inference by exchanging storage overhead for additional online communication incurred by transmitting plaintext matrices<ref:2606. p. 725349#pg2>. The security analysis confirms that the protocols securely compute the desired outputs under the standard semi-honest model and INDCPA assumption of RLWE<ref:2606. p. 825349#pg2>.

Cost advantage demonstrated

Analytical evaluations demonstrate that the proposed techniques significantly reduce online key-switch overhead and provide flexible trade-offs between storage and communication without requiring modifications to existing packing strategies<ref:2606.25349#pg2>. The results show that algorithms significantly reduce or completely eliminate online key-switching operations, which are among the most expensive operations in RLWE-based secure inference systems<ref:2606. p. 325349#pg2>. Overall, the method provides a flexible mechanism for balancing computation, storage, and communication costs while preserving the low-latency characteristics of online secure Transformer inference<ref:2606. p. 025349#pg2>.

Conclusion

The proposed method can be applied to both FHE-based and hybrid FHE–MPC secure Transformer inference<ref:2606.25349#pg2>. In noninteractive FHE-based inference, the algorithms can be incorporated into the first Transformer layer, while in hybrid FHE–MPC inference, they can be extended to all layers<ref:2606. p. 825349#pg2>. The proposed method reduces latency-critical cryptographic operations while preserving compatibility with existing secure Transformer inference systems<ref:2606. p. 125349#pg2>. The packing-independent reduction of inference-phase key switching overhead provides a complementary direction for improving the practicality of privacy-preserving Transformer inference<ref:2606. p. 025349#pg2>.

TABLE II summarizes the impact of the proposed algorithms on key-switching operations, preprocessing storage, and online communication costs for BLB and Arion<ref:2606. p. 025349#pg2> The storage–communication trade-offs discussed in this paper provide practical mechanisms for mitigating this overhead<ref:2606. p. 025349#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Preprocessing-assisted attention computation reduces online key-switching operations by decomposing attention into precomputable operations and online interactions, allowing for a packing-independent method for secure attention computation. This enables reduced latency in the inference phase without requiring modifications to existing packing strategies.

  2. Fused key-switch technique targets the multiplication-followed-by-rotation pattern in RNS CKKS matrix multiplication by combining relinearization and rotation into a single procedure, which reduces the associated computation costs. This optimization is independent of existing packing methods and can be integrated into current secure Transformer inference systems.

  3. Storage communication trade-off techniques replace large precomputed ciphertexts with modest online communication, enabling flexible deployment under varying resource constraints by replacing storage overhead with online transmission of auxiliary values like B1 + B2, J, and P. This balances the need for reduced key-switching overhead against server memory limitations in both FHE and hybrid MPC systems.

Sources

Related papers