SCOPE: Scalable Training-Free Semantic Coreset Selection for Long-Tailed Federated Learning

Scientific discovery increasingly relies on federated learning over data from high-resolution instruments, and these data are strongly non-IID and long-tailed. Coreset selection could shorten training, but existing federated coreset methods either prune with local heuristics that ignore the global class distribution, or first train a warm-up model at every client and upload class centroids of its features. This makes coreset selection a distributed bottleneck that consumes GPU time at every client, grows with the model and the number of clients, and must be repeated for every architecture. We introduce SCOPE, named for Semantic Coreset using Orthogonal Projection Embeddings, a scalable, training-free federated coreset framework for long-tailed data. Each client scores its samples once with a frozen, compact vision-language model and uploads scalar statistics per class. The server merges these into a global class profile, which lets each client remove anomalies and prune redundant samples only from classes that are locally abundant but globally common. SCOPE therefore requires no warm-up training, needs a single exchange with the server, and uploads a payload that depends only on the number of classes. On GPU nodes of an HPC cluster, with four datasets, including scientific micrographs, and six baselines, SCOPE selects its coreset up to 7.7x faster and uploads up to 293x less data than the feature-based baseline. As the number of clients grows from 10 to 500, SCOPE's time before training grows by fewer seconds than any baseline's, and at 500 clients the baselines need 1.7 to 3.6x SCOPE's time. Trained on the coreset, the global model reaches 98% of the full-data accuracy in 18 to 42% fewer global rounds than full-data training, with 8 to 21% less client computation over 200 rounds.