SSR: Sparse Segment Reduction for Ternary GEMM Acceleration
Large Language Models are resource-intensive, and Ternary Weight Networks (TWNs) offer high sparsity suitable for acceleration. Sparse Segment Reduction (SSR) is a new ternary matrix multiplication method that exploits these sparsity patterns using a specialized data format and computation trees. SSR outperforms existing methods like RSR++ by achieving significant speedups in inference and memory savings for ternary LLMs.