Democratizing MoE inference on commodity GPUs with CoMoE
CoMoE is a communication-efficient Mixture-of-Experts inference system designed to make large models run efficiently on low-bandwidth consumer GPUs. It solves the mismatch between high-end datacenter hardware and consumer hardware by using host memory as a smart routing hub, eliminating redundant data movement and synchronization stalls.