APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning
Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou, Brian Gelder, Ali Jannesari
cs.CV, cs.AI, cs.LG
Submitted: 2026-08-06
Comments: 22 pages, 7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices.
Terminology
Abstract
Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithms that are hard to apply across architectures. Uniform settings also ignore how differently individual layers respond to compression, which costs accuracy. We introduce APQF, an agentic profiling-guided framework that combines structured pruning, mixed-precision quantization-aware training, and accuracy recovery in one automated pipeline. A profiling agent measures how cost is distributed across the model and how sensitive each part is to pruning, and this evidence drives per-layer pruning ratios, per-layer bit-widths, and the recovery strategy, all proposed by LLM planners and validated before execution. To our knowledge, APQF is the first framework to combine LLM-guided, profiling-grounded decisions with a fully training-aware pruning and quantization pipeline for both CNNs and vision transformers. We evaluate APQF on ResNet, VGG7, ViT, DeiT, and Swin using ImageNet-1k and CIFAR-10. On ImageNet it cuts compute to 5.6-7.7 percent of the original bit-operations, a 13-18x reduction, while keeping accuracy close to the baseline, and under a 200K-image budget it stays roughly 17 points higher in Top-1 than existing joint pruning and quantization methods. On CIFAR-10 it compresses further than that method on four of five architectures. On VGG7 it reaches 93.15 percent using only 0.41 percent of baseline bit-operations, the only method at that compression level to improve on its full-precision baseline. Ablations show that uniform compression loses the most accuracy at matched compute, and that withholding profiling data from the planner hurts every model. Six LLM planners, including free open-weight ones, all reach 97.4-97.9 percent on Swin-Tiny.
Sources
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Pruning Filters for Efficient ConvNets
- Ternary Weight Networks
- Learning Discrete Weights Using the Local Reparameterization Trick
- Relaxed Quantization for Discretized Neural Networks
- Training and Inference with Integers in Deep Neural Networks
- Mixed Precision DNNs: All you need is a good parametrization
- Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
- Parameter-Efficient Fine-Tuning for Foundation Models
- EfficientLLM: Efficiency in Large Language Models
- A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
- LoRA+: Efficient Low Rank Adaptation of Large Models
- Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools
- Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios
- ProfilingAgent: Profiling-Guided Agentic Reasoning for Adaptive Model Optimization
- LLMs can Compress LLMs: Adaptive Pruning by Agents
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Very Deep Convolutional Networks for Large-Scale Image Recognition
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models