MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search

arXiv:2603.00720 · cs.LG · Submitted 2026-02-28 · Read on arXiv

cs.LG

Submitted: 2026-02-28

Updated: 2026-10-13

Comments: 29 pages; Project Page: this https URL: https://minkyoungcho.github.io/mars/

Project page: https://minkyoungcho.github.io/mars

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Adaptation — 2× in this paper · explained in 1 episode
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 13 episodes
• Multimodal Adaptive Rank Search — 1× in this paper
• Scaling Laws — 1× in this paper · explained in 4 episodes

Sources

• Qwen2.5-VL Technical Report
• LoRA Learns Less and Forgets Less
• Are We on the Right Way for Evaluating Large Vision-Language Models?
• Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
• GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
• Mordal: Automated Pretrained Model Selection for Vision Language Models
• Training Compute-Optimal Large Language Models
• Efficient Distributed MLLM Training with Cornstarch
• Scaling Laws for Neural Language Models
• OpenVLA: An Open-Source Vision-Language-Action Model
• LLaVA-OneVision: Easy Visual Task Transfer
• Evaluating Object Hallucination in Large Vision-Language Models
• On the Convergence Rate of LoRA Gradient Descent
• GeoLoRA: Geometric integration for parameter efficient fine-tuning
• Scaling Laws for Native Multimodal Models
• LoRA vs Full Fine-tuning: An Illusion of Equivalence
• Qwen2 Technical Report
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks