MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model
Zijun Jiang, Yangdi Lyu
cs.LG
Submitted: 2026-08-07
Updated: 2026-08-10
Comments: 14 pages, 9 figures, under review
Code: https://github.com/HKUSTGZ-MICS-LYU/MiCo-python
License: http://creativecommons.org/licenses/by/4.0/
The gist: Quantized Neural Networks (QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices.
Terminology
Abstract
Quantized Neural Networks (QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices. To mitigate accuracy degradation while maximizing speedup, layer-wise mixed-precision quantization (MPQ) becomes a popular solution. However, existing algorithms for exploring MPQ schemes are limited in flexibility and efficiency. Comprehending the complex impacts of different MPQ schemes on post-training quantization and quantization-aware training results is a challenge for conventional methods. Furthermore, an end-to-end framework for the optimization and deployment of MPQ models is missing in existing work. To address these challenges, we propose the MiCo framework, a holistic MPQ exploration and deployment framework for edge AI applications. The framework adopts a novel optimization algorithm to search for accuracy-optimal quantization configurations under strict latency constraints. We further extended the framework to MiCoPro, which introduces a robust Hardware-Aware Proxy (HAP) model to enhance prediction accuracy and hardware versatility. By leveraging target-specific latency modeling, MiCoPro enables rapid exploration and direct deployment from PyTorch models to bare-metal C code. We demonstrate the versatility of our framework on both the BitFusion accelerator and SIMD-extended RISC-V processors, achieving up to 40% of latency reduction with less than 3% of accuracy drop.
Sources
- CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations
- GP-NAS-ensemble: a model for NAS Performance Prediction
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks