DIVER: Diving Deeper into Distilled Data via Expressive Semantic Recovery
DIVER improves dataset distillation by using a dual-stage approach. Stage I creates a distilled dataset, and Stage II uses a pre-trained diffusion model to refine it into a synthetic dataset. This refinement recovers high-level semantics lost due to architecture patterns, significantly boosting the model's ability to generalize across different neural network types.