Towards an Analytical Definition of Sufficient Data
The study investigated which training samples are most informative for classification by analyzing their position relative to class centroids in reduced dimensional space. It found that samples closer to a class centroid provide less information than those further away. Excluding data nearest the centroid generally did not significantly harm accuracy across five datasets, suggesting a method for identifying highly informative, distant samples.