Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation
cs.LG, cs.CL
Submitted: 2026-07-02
Updated: 2026-08-31
Comments: To appear in EMNLP 2026. 17 pages, 7 figures, 8 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Calibrating LLMs with Information-Theoretic Evidential Deep Learning
- Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
- Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
- Dynamic Low-Rank Sparse Adaptation for Large Language Models
- Post-Optimization Adaptive Rank Allocation for LoRA
- DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
- Teaching Models to Express Their Uncertainty in Words
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs
- Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering
- Auto-Encoding Variational Bayes
- Categorical Reparameterization with Gumbel-Softmax
- The Llama 3 Herd of Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Measuring Massive Multitask Language Understanding
- Uncertainty quantification in fine-tuned LLMs using LoRA ensembles
- LoRA ensembles for large language model fine-tuning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks