When More Memory Fails: Hidden Feasibility Holes in Dynamic Tensor Rematerialization
Dynamic Tensor Rematerialization (DTR) trains neural networks by evicting tensors when memory limits are hit. This study found 'feasibility holes': memory budgets where DTR fails, even though slightly smaller and larger budgets succeed. This non-monotone behavior occurs because the eviction policy's cost-based score lacks a necessary mathematical property.