3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
3BASiL introduces an efficient one-shot method for decomposing Large Language Models (LLMs) into sparse and low-rank components. It uses a novel 3-Block Alternating Direction Method of Multipliers (ADMM) to minimize reconstruction error, combined with a transformer-matching refinement step. This combination achieves state-of-the-art compression results and faster runtime compared to existing methods.