AlterNet: Adaptive-Depth Mixture-of-Experts with Learned Halting for Elastic Inference
cs.AI
Submitted: 2026-11-05
Updated: 2026-11-05
Terminology
Sources
- PonderNet: Learning to Ponder
- LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- Adaptive Computation Time for Recurrent Neural Networks
- Distilling the Knowledge in a Neural Network
- Training Compute-Optimal Large Language Models
- Mixtral of Experts
- Sparse Layers are Critical to Scaling Looped Language Models
- Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
- AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth
- Qwen2.5 Technical Report
- OPT: Open Pre-trained Transformer Language Models
- Scaling Latent Reasoning via Looped Language Models
- ST-MoE: Designing Stable and Transferable Sparse Expert Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents