Asynchronous Heavy-Tailed Optimization
cs.LG
Submitted: 2026-02-20
Updated: 2026-10-28
Comments: 9-page main body, 25-page appendix, 5 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Making Pre-trained Language Models Better Few-shot Learners
- HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
- Efficient Distributed Optimization under Heavy-Tailed Noise
- Asynchronous Local-SGD Training for Language Modeling
- An Image is Worth 16x16 Words, What is a Video Worth?
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- Asynchronous Training Schemes in Distributed Learning with Time Delay
- FADAS: Towards Federated Adaptive Asynchronous Optimization
- Asynchronous Federated Optimization
- Asynchronous Stochastic Gradient Descent with Delay Compensation
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning