Pushing CPU Speech Synthesis to the Wall: Extreme Inference Tuning under Serverless Architecture and Billing
The paper optimized CPU speech synthesis inference for serverless billing by restructuring serving around instance lifetimes instead of always-on models. By using request-sized concurrency and a reclaimable instance lifecycle, they significantly reduced CPU and memory costs, achieving a 4.1x cost reduction compared to PyTorch defaults on the Kokoro-82M model.