Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime
cs.LG, cs.AI, cs.CL
Submitted: 2026-11-02
Updated: 2026-11-02
Comments: 32 pages, 22 figures
Code: https://github.com/arahmani1/library-of-layers
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- Universal Transformers
- Looped Transformers for Length Generalization
- Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
- Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
- UniPool: Learning Expert-to-Layer Ownership from Brief Global Access
- Do Transformers Need Three Projections? Systematic Study of QKV Variants
- Adam: A Method for Stochastic Optimization
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
- DeepSeek-V3 Technical Report
- Decoupled Weight Decay Regularization
- DataDecide: How to Predict Best Pretraining Data with Small Experiments
- Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
- Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
- Olmo 3
- gpt-oss-120b & gpt-oss-20b Model Card
- FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis