vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains
cs.DC, cs.LG
Submitted: 2026-09-29
Updated: 2026-09-29
Code: https://github.com/AKafakA/sglang-vskipper
Sources
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- Punica: Multi-Tenant LoRA Serving
- Training Verifiers to Solve Math Word Problems
- SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
- Dynamic Rebatching for Efficient Early-Exit Inference with DREX
- Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy
- LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- DASH: Input-Aware Dynamic Layer Skipping for Efficient LLM Inference with Markov Decision Policies
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Democratizing MoE inference on commodity GPUs with CoMoE