Training-Free Dynamic Upcycling of Expert Language Models
cs.LG, cs.CL
Submitted: 2026-03-31
Updated: 2026-11-18
Code: https://github.com/gensyn-ai/dume
Project page: http://skylion007.github.io/OpenWebTextCorpus
Terminology
Sources
- Towards Model Agnostic Federated Learning Using Knowledge Distillation
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- Training Compute-Optimal Large Language Models
- Editing Models with Task Arithmetic
- Mixtral of Experts
- Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- M2D2: A Massively Multi-domain Language Modeling Dataset
- Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
- LLaMA: Open and Efficient Foundation Language Models
- Schedule-Robust Online Continual Learning
- Magicoder: Empowering Code Generation with OSS-Instruct
- Instruction-Following Evaluation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis