TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel
The episode discusses the paper TaLK, which addresses the massive computational cost of jointly training Language Models and Graph Neural Networks on large datasets. TaLK uses a distillation approach, creating a small synthetic dataset to capture collective knowledge. This allows it to achieve 97% of full-dataset performance while drastically reducing training time, resulting in more efficient AI systems.