Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
Dooly profiles LLM inference operations configuration-agnostically by analyzing how input dimensions are derived from model settings or requests. It achieves redundancy awareness by selectively profiling only unique operations, significantly reducing profiling costs while maintaining high fidelity in simulating serving latency and scheduling decisions across different hardware and software stacks.