Know What You Need: Efficient Activation Checkpointing
cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/da462/KEAP
Terminology
Sources
- DeepSeek-V3 Technical Report
- The Llama 3 Herd of Models
- Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
- Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression
- Overcoming the Communication-Performance Tradeoff in LLM Pretraining
- Model Parallelism With Subnetwork Data Parallelism
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis