Terastal: Layer-Variant-based Scheduling for Real-Time Multi-DNN Workloads on Heterogeneous Accelerators
Terastal introduces a framework that combines offline design and online scheduling to improve real-time performance for multi-DNN workloads on different hardware accelerators. It creates 'layer variants' to reduce execution time on less ideal chips, uses virtual budgets to manage layer deadlines, and employs an intelligent scheduler that maps tasks and selects variants simultaneously, significantly cutting deadline misses while controlling accuracy loss.