Nereus: Adaptive Parallelism for LLM Post-Training
cs.DC, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/ROCm/rocm-systems
Sources
- AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training
- The SCIP Optimization Suite 10.0
- The Llama 3 Herd of Models
- Qwen3 Technical Report
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library
- LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
- DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
- StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
- APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Democratizing MoE inference on commodity GPUs with CoMoE