Hard-ReLU Gradient Descent Selects an Event-Free Sensitivity Limit
The episode discusses the paper "Hard-ReLU Gradient Descent Selectively Finds an Event-Free Sensitivity Limit." It explains how discrete fixed-step training processes fail to account for discontinuous event times, or saltations. This causes the calculated sensitivity of a model to diverge from its true continuous dynamics, potentially leading to inaccurate performance evaluations.