Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

Name: Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
Uploaded: 2026-10-17T19:58:46.167709+00:00
Duration: 287 s
Description: Gradient-based Language Model Pruner (GBLM-Pruner) is a novel, training-free method for pruning large language models using first-order Taylor expansion terms of gradients. It determines weight importance by combining normalized gradients and feature activations, outperforming prior methods like SparseGPT and Wanda across LLaMA and Vision Transformer models.

Gradient-based Language Model Pruner (GBLM-Pruner) is a novel, training-free method for pruning large language models using first-order Taylor expansion terms of gradients. It determines weight importance by combining normalized gradients and feature activations, outperforming prior methods like SparseGPT and Wanda across LLaMA and Vision Transformer models.

Open the full episode