Localized LoRA-MoE: Mixture of Low-Rank Experts With Adaptive Routing
cs.LG, cs.AI, cs.CL
Submitted: 2026-07-06
Updated: 2026-11-17
Terminology
Sources
- Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
- DeLoRA: Decoupling Angles and Strength in Low-rank Adaptation
- LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
- General Table Question Answering via Answer-Formula Joint Generation
- GoRA: Gradient-driven Adaptive Low Rank Adaptation
- LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
- VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
- DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution
- LoLDU: Low-Rank Adaptation via Lower-Diag-Upper Decomposition for Parameter-Efficient Fine-Tuning
- SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
- Hyperbolic Fine-Tuning for Large Language Models
- MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
- BoHA: Blockwise Hadamard Product Adaptation for Parameter-Efficient Fine-Tuning
- Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs
- LoR2C : Low-Rank Residual Connection Adaptation for Parameter-Efficient Fine-Tuning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis