OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction
OPTIMA introduces a method for one-shot post-training pruning of large language models by reformulating weight reconstruction as shared-Hessian column-wise Quadratic Programs (QPs). This approach balances accuracy and scalability by allowing per-column global optimality and massive parallel processing on GPUs, achieving up to 3.97% absolute accuracy improvement.