PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation

Name: PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation
Uploaded: 2026-10-03T07:25:54.939381+00:00
Duration: 349 s
Description: PRILoRA is a new method for fine-tuning large models called LoRA. It improves performance by combining two techniques: linearly increasing low-rank allocations across layers and ongoing pruning of the matrices based on input statistics. This yields state-of-the-art results on eight GLUE benchmarks without increasing trainable parameters or training time compared to standard LoRA.

PRILoRA is a new method for fine-tuning large models called LoRA. It improves performance by combining two techniques: linearly increasing low-rank allocations across layers and ongoing pruning of the matrices based on input statistics. This yields state-of-the-art results on eight GLUE benchmarks without increasing trainable parameters or training time compared to standard LoRA.

Open the full episode