A Survey on Collaborative Mechanisms Between Large and Small Language Models
cs.AI, cs.CL
Submitted: 2025-05-12
Updated: 2026-11-18
Terminology
Sources
- Self-Resource Allocation in Multi-Agent LLM Systems
- Knowledge-Decoupled Synergetic Learning: An MLLM based Collaborative Approach to Few-shot Multimodal Dialogue Intention Recognition
- Accelerating Large Language Model Decoding with Speculative Sampling
- Harnessing Multiple Large Language Models: A Survey on LLM Ensemble
- Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalization
- Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
- Sliding-Window Merging for Compacting Patch-Redundant Layers in LLMs
- Hymba: A Hybrid-head Architecture for Small Language Models
- Federated Co-tuning Framework for Large and Small Language Models
- Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration
- VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
- DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis
- LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
- MiniLLM: On-Policy Distillation of Large Language Models
- Bounding and Filling: A Fast and Flexible Framework for Image Captioning
- KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations
- Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
- Auxiliary task demands mask the capabilities of smaller language models
- Dynamic Low-Rank Sparse Adaptation for Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents