Systematic Exploration of Multi-core Architectures for Efficient LLM Serving using WaferAI-SIM
This study systematically explored multi-core NPU architectures for LLM serving by developing a hybrid simulation framework called NpuSim. It analyzed how tensor parallelism, core placement, and memory management affect performance across different hardware setups. The findings guide system design toward specific strategies like heterogeneous PD disaggregation for prefill workloads to achieve significant speedups.