LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights

Name: LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
Uploaded: 2026-08-20T13:22:21.925398+00:00
Duration: 643 s
Description: The episode reviews LORA-CRAFT, a method for efficiently fine-tuning large language models. It adapts pre-trained attention weights across layers using Tucker decomposition. By freezing most weights and training only tiny adjustment matrices, CRAFT achieves competitive performance with drastically fewer trainable parameters than standard methods.

The episode reviews LORA-CRAFT, a method for efficiently fine-tuning large language models. It adapts pre-trained attention weights across layers using Tucker decomposition. By freezing most weights and training only tiny adjustment matrices, CRAFT achieves competitive performance with drastically fewer trainable parameters than standard methods.

Open the full episode