PithTrain: A Compact and Agent-Native MoE Training System
cs.LG, cs.AI, cs.CL, cs.DC
Submitted: 2026-05-29
Updated: 2026-11-07
Code: https://github.com/mlc-ai/pith-train
Terminology
Sources
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
- DataComp-LM: In search of the next generation of training sets for language models
- MoBA: Mixture of Block Attention for Long-Context LLMs
- Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
- Differential Transformer
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis