DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching
cs.DC, cs.AI, cs.LG, cs.SY, eess.SY
Submitted: 2026-10-06
Updated: 2026-10-06
Code: https://github.com/Large-scale-Sustainable-Computing-LSC/dysco-artifact
Sources
- MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
- SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization
- DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
- EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
- P/D-Device: Disaggregated Large Language Model between Cloud and Devices
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- WildChat: 1M ChatGPT Interaction Logs in the Wild
- LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Democratizing MoE inference on commodity GPUs with CoMoE