A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers
The paper introduces a new benchmark to rigorously test deep learning image classifiers by evaluating performance across five distinct data types: clean, corrupt, adversarial, novel classes, and unrecognisable images. It proposes a single metric called detection accuracy rate (DAR) to provide a consistent measure of overall robustness and reveals trade-offs where training for one type of robustness can negatively impact performance on others.