Class Density and Dataset Quality in High-Dimensional, Unstructured Data
The authors defined class density using a mean-derived calculation to measure sample similarity within classes in high-dimensional data. They also proposed a dataset quality metric based on the variance of these class densities, suggesting datasets with a quality score above 10 are suitable for redundant data removal. Experiments showed this approach works well for most datasets, though complex cases like Imagenette require caution.