A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
cs.AI
Submitted: 2024-08-02
Updated: 2026-11-01
Terminology
Sources
- A Survey on Multimodal Large Language Models
- MM-LLMs: Recent Advances in MultiModal Large Language Models
- Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
- Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports
- BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
- Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
- GPT-4 Technical Report
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
- Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
- Gemini: A Family of Highly Capable Multimodal Models
- NExT-GPT: Any-to-Any Multimodal LLM
- Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models
- VideoChat: Chat-Centric Video Understanding
- Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
- SALMONN: Towards Generic Hearing Abilities for Large Language Models
- Understanding LLMs: A Comprehensive Overview from Training to Inference
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages
- BEATs: Audio Pre-Training with Acoustic Tokenizers
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents