One Model, Multiple Compute Budgets: Multi-Ratio ConceptMoE for Flexible Compute Across Turns
cs.LG, cs.AI
Submitted: 2026-10-08
Updated: 2026-10-08
Sources
- Program Synthesis with Large Language Models
- Longformer: The Long-Document Transformer
- Flextron: Many-in-One Flexible Large Language Model
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- Evaluating Large Language Models Trained on Code
- On the Measure of Intelligence
- Depth-Adaptive Transformer
- ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure
- TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
- CodeChat-Eval: Evaluating Large Language Models in Multi-Turn Code Refinement Dialogues
- Measuring Massive Multitask Language Understanding
- Measuring Mathematical Problem Solving With the MATH Dataset
- ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation
- Fast Inference from Transformers via Speculative Decoding
- From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Humanity's Last Exam
- Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space
- Compressive Transformers for Long-Range Sequence Modelling
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks