CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference
summary
The gist
MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters However…
In short
CrypTorch is a compiler framework designed to improve accuracy and performance in MPC-based machine learning by automatically selecting transformations. It addresses limitations in existing frameworks by making transformations modular, allowing users to inspect and replace them easily. This auto-tuning capability yields significant speedups, up to 8.32x end-to-end, balancing the trade-off between speed and model quality.
Key concepts
- Multi-Party Computation (MPC)
- MPC allows multiple parties to jointly run a machine learning workload without ever sharing their private data or model parameters with each other. It ensures privacy while enabling collaborative computation across different entities.
- Operator Approximation
- Since MPC cannot natively execute every operation in a neural network, operators must be approximated. This process involves selecting simpler mathematical substitutes for complex operations to make them runnable under MPC constraints, which can introduce errors.
- Auto-tuning Compiler
- CrypTorch features an automatic compiler that analyzes the model and iteratively tests different operator approximations on a standard GPU runtime. It selects the optimal combination of approximations to maximize speedup while minimizing the loss in the original machine learning program's accuracy.
Terminology used across episodes
This episode discusses
- CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference · Paper Radio
- Is Private Learning Possible with Instance Encoding?
- Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures
- Secure Evaluation of Quantized Neural Networks
- PriViT: Vision Transformers for Fast Private Inference
- PUMA: Secure Inference of LLaMA-7B in Five Minutes
- Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning
- Efficient Privacy-Preserving Machine Learning with Lightweight Trusted Hardware
- Cerium: A Multi-GPU Framework for Terabyte-Scale Encrypted Inference · Paper Radio
- DeepReShape: Redesigning Neural Networks for Efficient Private Inference
- HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks
- GPU-based Private Information Retrieval for On-Device Machine Learning Inference
- MPCFormer: fast, performant and private Transformer inference with MPC
- Understanding TEE Containers, Easy to Use? Hard to Trust
- SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC
- Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference
- AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference
- PAC Privacy: Automatic Privacy Measurement and Control of Data Processing
- Privacy-Preserving Instructions for Aligning Large Language Models
- TruncFormer: Private LLM Inference Using Only Truncations
The paper
CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference · Read on arXiv
Jinyu Liu, Gang Tan, Kiwan Maeng
The Pennsylvania State University
MPC-based ML uses secure multi-party computation (MPC) to execute machine learning (ML) workloads across multiple parties without requiring any party to reveal its private data or model parameters. Our characterization study finds that the performance of state-of-the-art MPC frameworks is often dominated by operator-approximation strategies, where ML operators not natively supported by MPC are replaced with iterative methods or polynomials. These approximations are typically chosen heuristically by each framework and applied globally. As a result, they can be unnecessarily expensive in some cases, yet insufficiently accurate in others. We find that the best approximation choice depends on the context, target framework, and application, making a single globally fixed approximation fundamentally suboptimal. To address this problem, we design CrypTorch, a multi-stage compiler for MPC-based ML that supports multiple MPC frameworks as backends. At the core of CrypTorch is its auto-tuning stage, which extracts cost estimates from the target MPC backend and quality estimates from a proxy PyTorch execution, and uses them to automatically select a set of approximations that maximizes performance while satisfying the given quality constraint. We show that CrypTorch's auto-tuner improves performance across multiple MPC backends by 1.14--1.60x without degrading accuracy, and by 1.17--1.92x with less than 3% accuracy degradation. CrypTorch's unified design also improves programmability and usability by allowing multiple backends to share other common compiler functionality and optimizations. We further show that the auto-tuner extends to a related fully homomorphic encryption (FHE) backend, reducing the number of bootstrapping operations by 1.31--1.65x.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference".
Elias: Here's the gist of the paper. MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters However, existing frameworks frequently degrade accuracy and performance due to a series of MPCspecific transformations that adds errors and overheads CrypTorch proposes a modular, extensible, and iteratively-testable compiler framework for MPC-based ML It automatically chooses a set of transformations to balance performance and accuracy during the operator approximation stage CrypTorch’s auto-tuning alone provides 3.74–8.32× combined speedup end-to-end for complex models Problem Identification and Existing Limitations Existing MPC-based ML frameworks apply transformations in a manner entirely opaque to users, with little visibility into which transformations were applied or how they affect performance/accuracy Issues identified include: "Lack of Extensibility to New Ideas" Frameworks often suffer from little modularity or extensibility, making it difficult to adopt new, competitive ideas "Inability to Debug Accuracy Issues" When accuracy degrades non-negligibly, it is hard to pinpoint the source of error because all MPC-related transformations are applied together "Suboptimal Performance Due to Global and Heuristic Operator Approximations" The major overhead comes from the comparison and multiplication used to approximate operators, constituting 72.8–96% of the overhead in complex models CrypTorch Framework Design CrypTorch is designed to overcome identified issues by fulfilling requirements R1 through R4 "R3 It must automatically find a set of approximations that balances performance and accuracy" The framework decomposes MPC-specific transformations into multiple compilation stages, allowing users to easily inspect and replace each transformation CrypTorch’s operator approximation stage automatically chooses approximations for operators that MPC cannot run natively, quickly testing each candidate on an efficient (non-MPC) GPU runtime The auto-tuner uses a selected search strategy to automatically find a set of approximations that maximizes performance while minimizing the target ML program’s quality loss Performance Gains and Auto-tuning Efficacy Adopting state-of-the-art practices allows existing frameworks to achieve 14.4–96.2× peroperator speedup and 6.4–11.1× end-to-end speedup CrypTorch’s auto-tuning achieves an additional 3.74–8.32× speedup over LAN and 4.18–6.84× over WAN compared to the original CrypTen The auto-tuner results show that CrypTorch-aggressive achieves 1.35–1.74× speedup on WAN, and 3.74–8.32× speedup over LAN compared to the original CrypTen Extensibility and Debuggability in Practice CrypTorch demonstrates superior extensibility by allowing users to add new approximations through a programming interface Adding LayerNorm support required only 3 lines using the pass-writing interface, compared to 46 lines of code modification in CrypTen The iterative debugging feature allows easier debugging because it produces an intermediate graph that can be tested iteratively on a regular PyTorch runtime without any MPC The final output is a torch.fx.
Paper summary: Nadia: So we've been talking about CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference, and how it tries to make those heavy math approximations transparent and tunable <ref:2511.19711#pg1>.
Elias: Yeah, the authors are looking at existing MPC machine learning frameworks and they're trying to build a modular compiler that automatically tunes the specific math approximations needed for performance without messing up accuracy <ref:2511.19711#pg2>.
Priya: It’s about making the whole process clear so you can actually see what transformations are happening and why things are changing <ref:2511.19711#pg3>.
Nadia: The core problem they're addressing is that current frameworks use these hidden transformations that make it really hard to optimize things when you want to build something new <ref:2511.19711#pg4>.
Elias: CrypTorch splits the work into stages so users can check and fix each part individually <ref:2511.19711#pg3>.
Priya: They show how this transparency helps you see the trade-offs between speed and accuracy which is what matters in this space <ref:2511.19711#pg4>.
Nadia: And they are saying that existing frameworks have these opaque transformations that make optimization really difficult for anyone trying to build something new <ref:2511.19711#pg1>.
Elias: The framework is designed to be modular so you can add new ideas or approximations easily without rewriting everything from scratch <ref:2511.19711#pg3>.
Priya: They also show how easy it is to plug in new components, like adding LayerNorm support, which used way less work than in the older versions of this framework <ref:2511.19711#pg3>.
Nadia: It’s about taking a complex area and making the optimization process less mysterious and more manageable for anyone trying to use it practically <ref:2511.19711#pg3>.
Elias: And they get this clear output, a torch fx graph, which makes everything much easier to trace back when things go wrong <ref:2511.19711#pg3>.
Priya: This paper is all about CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference <ref:2511.19711#pg3>.
Conclusion: Nadia: So we've covered CrypTorch, which they call a unified PyTorch-based auto-tuning compiler for private machine learning inference, and how it tries to make those heavy math approximations transparent and tunable <ref:2511.19711#pg1>.
Elias: The authors are looking at existing MPC machine learning frameworks and they're trying to build a modular compiler that automatically tunes the specific math approximations needed for performance without messing up accuracy <ref:2511.19711#pg2>.
Priya: For the people using private machine learning, this means the data they get is more trustworthy because there’s an explicit way to keep the quality loss down while you chase those speed gains <ref:2511.19711#pg4>.
Nadia: It suggests that building private machine learning isn't just about implementing the security protocols themselves, but also having better tools on top of them to manage all that necessary optimization work <ref:2511.19711#pg4>.
Priya: It shows how easy it is to plug in new components, like adding LayerNorm support, which used way less work than in the older versions of this framework <ref:2511.19711#pg3>.
Elias: And the whole point is that you get this clear output, a torch fx graph, which makes everything much easier to trace back when things go wrong <ref:2511.19711#pg3>.
Priya: It moves the focus from just making it work to actually understanding why it works and where the trade-offs are happening between speed and accuracy <ref:2511.19711#pg3>.
Elias: It’s a tool designed to make the MPC compiler process transparent and tunable <ref:2511.19711#pg3>.
Priya: This suggests that the future of private machine learning isn't just about implementing the MPC protocols themselves, but also building better tools on top of them to manage all those necessary optimizations <ref:2511.19711#pg2>.
Elias: We'll get into how this actually works in the next segment, looking at those specific design choices <ref:2511.19711#pg3>.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18