Adaptive Utilization of Low-Rank Adaptation via Conditioned Gating
cs.CL
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: ICML 2026
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
- Training Verifiers to Solve Math Word Problems
- Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
- Towards a Unified View of Parameter-Efficient Transfer Learning
- Disentangled Contrastive Learning for Zero-Shot Multilingual Dense Retrieval
- The Power of Scale for Parameter-Efficient Prompt Tuning
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models
- Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
- Gemma: Open Models Based on Gemini Research and Technology
- Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment
- Qwen3 Technical Report
- Qwen2.5 Technical Report
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering