HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
The episode discusses HiAP, a framework for automatically pruning Vision Transformers to make them smaller and faster without losing accuracy. The hosts explain how HiAP uses learned gates and cost models to simultaneously learn an efficient architecture during training. They conclude that this method simplifies compression by making it part of the learning process.