AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning
cs.LG
Submitted: 2026-08-24
Updated: 2026-10-11
Comments: 23 pages, 5 figures, 19 tables. Code and experimental artifacts: https://github.com/Romyull-Islam/AQLoRA
Code: https://github.com/Romyull-Islam/AQLoRA
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic
- CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
- QLoRA: Efficient Finetuning of Quantized LLMs
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
- MobileFineTuner: A Mobile-Native Framework for On-Device LLM Fine-Tuning in Real-World Embedded AI Applications
- LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
- LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
- Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
- QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
- A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
- SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity
- LoRA Is Slower Than You Think
- QEFT: Quantization for Efficient Fine-Tuning of LLMs
- OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
- LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
- Parameter-Efficient Fine-Tuning without Introducing New Latency
- ApiQ: Finetuning of 2-Bit Quantized Large Language Model
- DoRA: Weight-Decomposed Low-Rank Adaptation
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks