Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review
stat.ML, cs.LG, stat.CO
Submitted: 2023-11-20
Updated: 2023-11-20
Terminology
Sources
- Theano: A Python framework for fast computation of mathematical expressions
- Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- MinimalRNN: Toward More Interpretable and Trainable Recurrent Neural Networks
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Regularized Binary Network Training
- The State of Sparsity in Deep Neural Networks
- A Survey on Methods and Theories of Quantized Neural Networks
- Distilling the Knowledge in a Neural Network
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
- Quantization for Rapid Deployment of Deep Neural Networks
- Neural networks on microcontrollers: saving memory at inference via operator reordering
- Convolutional Neural Networks using Logarithmic Data Representation
- An Overview of Neural Network Compression
- GPT-4 Technical Report
- Efficient Neural Network Deployment for Microcontroller
- TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference