CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference".
Elias: Here's the gist of the paper. MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters However, existing frameworks frequently degrade accuracy and performance due to a series of MPCspecific transformations that adds errors and overheads CrypTorch proposes a modular, extensible, and iteratively-testable compiler framework for MPC-based ML It automatically chooses a set of transformations to balance performance and accuracy during the operator approximation stage CrypTorch’s auto-tuning alone provides 3.74–8.32× combined speedup end-to-end for complex models Problem Identification and Existing Limitations Existing MPC-based ML frameworks apply transformations in a manner entirely opaque to users, with little visibility into which transformations were applied or how they affect performance/accuracy Issues identified include: "Lack of Extensibility to New Ideas" Frameworks often suffer from little modularity or extensibility, making it difficult to adopt new, competitive ideas "Inability to Debug Accuracy Issues" When accuracy degrades non-negligibly, it is hard to pinpoint the source of error because all MPC-related transformations are applied together "Suboptimal Performance Due to Global and Heuristic Operator Approximations" The major overhead comes from the comparison and multiplication used to approximate operators, constituting 72.8–96% of the overhead in complex models CrypTorch Framework Design CrypTorch is designed to overcome identified issues by fulfilling requirements R1 through R4 "R3 It must automatically find a set of approximations that balances performance and accuracy" The framework decomposes MPC-specific transformations into multiple compilation stages, allowing users to easily inspect and replace each transformation CrypTorch’s operator approximation stage automatically chooses approximations for operators that MPC cannot run natively, quickly testing each candidate on an efficient (non-MPC) GPU runtime The auto-tuner uses a selected search strategy to automatically find a set of approximations that maximizes performance while minimizing the target ML program’s quality loss Performance Gains and Auto-tuning Efficacy Adopting state-of-the-art practices allows existing frameworks to achieve 14.4–96.2× peroperator speedup and 6.4–11.1× end-to-end speedup CrypTorch’s auto-tuning achieves an additional 3.74–8.32× speedup over LAN and 4.18–6.84× over WAN compared to the original CrypTen The auto-tuner results show that CrypTorch-aggressive achieves 1.35–1.74× speedup on WAN, and 3.74–8.32× speedup over LAN compared to the original CrypTen Extensibility and Debuggability in Practice CrypTorch demonstrates superior extensibility by allowing users to add new approximations through a programming interface Adding LayerNorm support required only 3 lines using the pass-writing interface, compared to 46 lines of code modification in CrypTen The iterative debugging feature allows easier debugging because it produces an intermediate graph that can be tested iteratively on a regular PyTorch runtime without any MPC The final output is a torch.fx.
Paper summary: Nadia: So we've been talking about CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference, and how it tries to make those heavy math approximations transparent and tunable <ref:2511.19711#pg1>.
Elias: Yeah, the authors are looking at existing MPC machine learning frameworks and they're trying to build a modular compiler that automatically tunes the specific math approximations needed for performance without messing up accuracy <ref:2511.19711#pg2>.
Priya: It’s about making the whole process clear so you can actually see what transformations are happening and why things are changing <ref:2511.19711#pg3>.
Nadia: The core problem they're addressing is that current frameworks use these hidden transformations that make it really hard to optimize things when you want to build something new <ref:2511.19711#pg4>.
Elias: CrypTorch splits the work into stages so users can check and fix each part individually <ref:2511.19711#pg3>.
Priya: They show how this transparency helps you see the trade-offs between speed and accuracy which is what matters in this space <ref:2511.19711#pg4>.
Nadia: And they are saying that existing frameworks have these opaque transformations that make optimization really difficult for anyone trying to build something new <ref:2511.19711#pg1>.
Elias: The framework is designed to be modular so you can add new ideas or approximations easily without rewriting everything from scratch <ref:2511.19711#pg3>.
Priya: They also show how easy it is to plug in new components, like adding LayerNorm support, which used way less work than in the older versions of this framework <ref:2511.19711#pg3>.
Nadia: It’s about taking a complex area and making the optimization process less mysterious and more manageable for anyone trying to use it practically <ref:2511.19711#pg3>.
Elias: And they get this clear output, a torch fx graph, which makes everything much easier to trace back when things go wrong <ref:2511.19711#pg3>.
Priya: This paper is all about CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference <ref:2511.19711#pg3>.
Conclusion: Nadia: So we've covered CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference, and how it tries to make those heavy math approximations transparent and tunable <ref:2511.19711#pg1>.
Elias: The authors are looking at existing MPC machine learning frameworks and they're trying to build a modular compiler that automatically tunes the specific math approximations needed for performance without messing up accuracy <ref:2511.19711#pg2>.
Priya: For the people using private machine learning, this means the data they get is more trustworthy because there’s an explicit way to keep the quality loss down while you chase those speed gains <ref:2511.19711#pg4>.
Nadia: It suggests that building private machine learning isn't just about implementing the security protocols themselves, but also having better tools on top of them to manage all that necessary optimization work <ref:2511.19711#pg4>.
Priya: It shows how easy it is to plug in new components, like adding LayerNorm support, which used way less work than in the older versions of this framework <ref:2511.19711#pg3>.
Elias: And the whole point is that you get this clear output, a torch fx graph, which makes everything much easier to trace back when things go wrong <ref:2511.19711#pg3>.
Priya: It moves the focus from just making it work to actually understanding why it works and where the trade-offs are happening between speed and accuracy <ref:2511.19711#pg3>.
Elias: It’s a tool designed to make the MPC compiler process transparent and tunable <ref:2511.19711#pg3>.
Priya: This suggests that the future of private machine learning isn't just about implementing the MPC protocols themselves, but also building better tools on top of them to manage all those necessary optimizations <ref:2511.19711#pg2>.
Elias: We'll get into how this actually works in the next segment, looking at those specific design choices <ref:2511.19711#pg3>.
Jinyu Liu, Gang Tan, Kiwan Maeng
The Pennsylvania State University
cs.CR, cs.AI, cs.PL
Submitted: 2025-11-24
Updated: 2026-11-13
Comments: 29 pages, 18 figures. Revised for resubmission. Submitted to PLDI 2027
Code: https://github.com/secretflow/spu
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 87/100
The gist: MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters However…
Key concepts
- Multi-Party Computation (MPC)
- MPC allows multiple parties to jointly run a machine learning workload without ever sharing their private data or model parameters with each other. It ensures privacy while enabling collaborative computation across different entities.
- Operator Approximation
- Since MPC cannot natively execute every operation in a neural network, operators must be approximated. This process involves selecting simpler mathematical substitutes for complex operations to make them runnable under MPC constraints, which can introduce errors.
- Auto-tuning Compiler
- CrypTorch features an automatic compiler that analyzes the model and iteratively tests different operator approximations on a standard GPU runtime. It selects the optimal combination of approximations to maximize speedup while minimizing the loss in the original machine learning program's accuracy.
Terminology
Summary
MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters However, existing frameworks frequently degrade accuracy and performance due to a series of MPCspecific transformations that adds errors and overheads CrypTorch proposes a modular, extensible, and iteratively-testable compiler framework for MPC-based ML It automatically chooses a set of transformations to balance performance and accuracy during the operator approximation stage CrypTorch’s auto-tuning alone provides 3. p. 0 edu
Abstract
MPC-based ML uses multi-party computation (MPC) to run
machine learning (ML) workloads across multiple parties
without each having to share their private data or model
parameters. However, existing frameworks frequently degrade accuracy and performance due to a series of MPCspecific transformations that adds errors and overheads.Where the paper says it
Problem Identification and Existing Limitations
Existing MPC-based ML frameworks apply transformations in a manner entirely opaque to users, with little visibility into which transformations were applied or how they affect performance/accuracy Issues identified include:
Lack of Extensibility to New Ideas
Frameworks often suffer from little modularity or extensibility, making it difficult to adopt new, competitive ideas
Inability to Debug Accuracy Issues
When accuracy degrades non-negligibly, it is hard to pinpoint the source of error because all MPC-related transformations are applied together
Suboptimal Performance Due to Global and Heuristic Operator Approximations
The major overhead comes from the comparison and multiplication used to approximate operators, constituting 72.8–96% of the overhead in complex models
CrypTorch Framework Design
CrypTorch is designed to overcome identified issues by fulfilling requirements R1 through R4
R3 It must automatically find a set of approximations that balances performance and accuracy
The framework decomposes MPC-specific transformations into multiple compilation stages, allowing users to easily inspect and replace each transformation
CrypTorch’s operator approximation stage automatically chooses approximations for operators that MPC cannot run natively, quickly testing each candidate on an efficient (non-MPC) GPU runtime
The auto-tuner uses a selected search strategy to automatically find a set of approximations that maximizes performance while minimizing the target ML program’s quality loss
Performance Gains and Auto-tuning Efficacy
Adopting state-of-the-art practices allows existing frameworks to achieve 14.4–96.2× peroperator speedup and 6. p. 0 We show that adopting state-of-the-art practices (better
operator approximations, MPC protocols, and kernel implementations) allows an existing framework (we tested with
CrypTen [62], but many of our contributions are agnostic
to the underlying MPC runtime) to achieve 14. 2× peroperator speedup and 6.Where the paper says it
CrypTorch’s auto-tuning achieves an additional 3.74–8.32× speedup over LAN and 4. p. 9 50
LAN
EfficientNet ViT
BERT-QNLI
BERT-QQP
BERT-SST2
1. 75 Normalized speedup
WAN
CrypTen (optimized)
CrypTorch-eco (< 0%)
CrypTorch-moderate (< 2%)
CrypTorch-aggressive (< 5%)
Figure 11.Where the paper says it
The auto-tuner results show that CrypTorch-aggressive achieves 1.35–1.74× speedup on WAN, and 3. p. 9 50
LAN
EfficientNet ViT
BERT-QNLI
BERT-QQP
BERT-SST2
1. 75 Normalized speedup
WAN
CrypTen (optimized)
CrypTorch-eco (< 0%)
CrypTorch-moderate (< 2%)
CrypTorch-aggressive (< 5%)
Figure 11.Where the paper says it
Extensibility and Debuggability in Practice
CrypTorch demonstrates superior extensibility by allowing users to add new approximations through a programming interface
Adding LayerNorm support required only 3 lines using the pass-writing interface, compared to 46 lines of code modification in CrypTen
The iterative debugging feature allows easier debugging because it produces an intermediate graph that can be tested iteratively on a regular PyTorch runtime without any MPC
The final output is a torch.fx.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Modular compiler design allows users to easily inspect and optimize MPC-specific transformations by splitting them into modular compilation stages that emit an executable graph after each transformation. This enables iterative testing to pinpoint problematic transformations.
-
Automated operator approximation auto-tuning can choose a set of approximations from a pool of choices to balance performance and accuracy, achieving up to 1.21–1.5x speedup without accuracy loss, or 1.33–1.74x speedup when some degradation is allowed for complex models.
-
The compiler backend translates the intermediate representation into an MPC-executable graph by first converting each IR node to operate on an integer ring through proper encryption, scaling, and truncation before lowering operators to the relevant MPC or nonMPC runtime kernel.
-
CrypTorch enables per-layer granularity tuning of approximations, allowing users to tune parameters like self.t and self.clamp for operators such as exp or GELU based on context, which can be fixed or exposed to the auto-tuner.
-
The system can debug accuracy issues by generating an executable graph after each transformation stage, enabling iterative testing of model accuracy and pinpointing the source of error in existing frameworks where all MPC-related transformations are applied together.
Abstract
MPC-based ML uses secure multi-party computation (MPC) to execute machine learning (ML) workloads across multiple parties without requiring any party to reveal its private data or model parameters. Our characterization study finds that the performance of state-of-the-art MPC frameworks is often dominated by operator-approximation strategies, where ML operators not natively supported by MPC are replaced with iterative methods or polynomials. These approximations are typically chosen heuristically by each framework and applied globally. As a result, they can be unnecessarily expensive in some cases, yet insufficiently accurate in others. We find that the best approximation choice depends on the context, target framework, and application, making a single globally fixed approximation fundamentally suboptimal. To address this problem, we design CrypTorch, a multi-stage compiler for MPC-based ML that supports multiple MPC frameworks as backends. At the core of CrypTorch is its auto-tuning stage, which extracts cost estimates from the target MPC backend and quality estimates from a proxy PyTorch execution, and uses them to automatically select a set of approximations that maximizes performance while satisfying the given quality constraint. We show that CrypTorch's auto-tuner improves performance across multiple MPC backends by 1.14--1.60x without degrading accuracy, and by 1.17--1.92x with less than 3% accuracy degradation. CrypTorch's unified design also improves programmability and usability by allowing multiple backends to share other common compiler functionality and optimizations. We further show that the auto-tuner extends to a related fully homomorphic encryption (FHE) backend, reducing the number of bootstrapping operations by 1.31--1.65x.
Sources
- Is Private Learning Possible with Instance Encoding?
- Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures
- Secure Evaluation of Quantized Neural Networks
- PriViT: Vision Transformers for Fast Private Inference
- PUMA: Secure Inference of LLaMA-7B in Five Minutes
- Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning
- Efficient Privacy-Preserving Machine Learning with Lightweight Trusted Hardware
- Cerium: A Multi-GPU Framework for Terabyte-Scale Encrypted Inference
- DeepReShape: Redesigning Neural Networks for Efficient Private Inference
- HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks
- GPU-based Private Information Retrieval for On-Device Machine Learning Inference
- MPCFormer: fast, performant and private Transformer inference with MPC
- Understanding TEE Containers, Easy to Use? Hard to Trust
- SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC
- Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference
- AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference
- PAC Privacy: Automatic Privacy Measurement and Control of Data Processing
- Privacy-Preserving Instructions for Aligning Large Language Models
- TruncFormer: Private LLM Inference Using Only Truncations
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data