Surprisingly High Redundancy in Electronic Structure Data Across Materials Explained by Low Intrinsic Dimensionality
The research found significant redundancy in large electronic structure datasets across materials because this data lives on a low-dimensional, non-linear manifold. By using coverage-centric pruning strategies, researchers can reduce dataset sizes by up to 99% while maintaining high accuracy and generalizability. This suggests that minimal representative datasets are sufficient for material science machine learning.