FRESCO: A Novel Consistency Control for Asynchronous Pipeline Parallel Training
cs.LG, cs.AI, cs.DC
Submitted: 2026-11-08
Updated: 2026-11-08
Code: https://github.com/LOG-postech/fresco
Project page: http://skylion007.github.io/OpenWebTextCorpus
Terminology
Sources
- Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform
- Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning
- The Llama 3 Herd of Models
- Logarithmic-time Schedules for Scaling Language Models with Momentum
- Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees
- AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
- XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
- Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
- TRGP: Trust Region Gradient Projection for Continual Learning
- DeepSeek-V3 Technical Report
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Kimi K2: Open Agentic Intelligence
- Orthogonal Calibration for Asynchronous Federated Learning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis