Private Collaborative Edge Inference via Over-the-Air Computation
cs.LG, cs.AI, cs.CR, cs.IT, math.IT
Submitted: 2024-07-30
Updated: 2025-01-14
Comments: 17 pages, 8 figures. This work extends from our preliminary study presented at the 2022 IEEE International Symposium on Information Theory [1]. arXiv admin note: text overlap with arXiv:2202.03129
Journal ref: IEEE Trans. Mach. Learn. Commun. Netw. 3, 215-231 (2025)
DOI: 10.1109/TMLCN.2025.3526551
Code: https://github.com/ipc-lab/collaborative-inference-oac
License: http://creativecommons.org/licenses/by/4.0/
The gist: We consider collaborative inference at the wireless edge, where each client's model is trained independently on its local dataset.
Terminology
Abstract
We consider collaborative inference at the wireless edge, where each client's model is trained independently on its local dataset. Clients are queried in parallel to make an accurate decision collaboratively. In addition to maximizing the inference accuracy, we also want to ensure the privacy of local models. To this end, we leverage the superposition property of the multiple access channel to implement bandwidth-efficient multi-user inference methods. We propose different methods for ensemble and multi-view classification that exploit over-the-air computation (OAC). We show that these schemes perform better than their orthogonal counterparts with statistically significant differences while using fewer resources and providing privacy guarantees. We also provide experimental results verifying the benefits of the proposed OAC approach to multi-user inference, and perform an ablation study to demonstrate the effectiveness of our design choices. We share the source code of the framework publicly on Github to facilitate further research and reproducibility.
Sources
- Differentially Private Federated Learning: A Client Level Perspective
- Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints
- Federated Learning: Strategies for Improving Communication Efficiency
- Features-over-the-Air: Contrastive Learning Enabled Cooperative Edge Inference
- Over-the-Air Multi-View Pooling for Distributed Sensing
- Composition of Differential Privacy & Privacy Amplification by Subsampling
- Why Does Differential Privacy with Large Epsilon Defend Against Practical Membership Inference Attacks?
- How to generate random matrices from the classical compact groups
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Differential Privacy with Random Projections and Sign Random Projections
- Waveforms for Computing Over the Air
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks