Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval

The two-tower model is widely used in the retrieval stage of large-scale recommendation systems, where training typically relies on in-batch and/or out-of-batch negative sampling. These methods, however, tend to produce easy negatives that the model learns quickly and that provide little training signal. This paper proposes a self-supervised, cluster-based hard negative sampling technique that draws negatives from the same semantic cluster as the positive item; in our production deployment the clusters are derived from large language model (LLM) based multimodal content representations, so that intra-cluster items are genuinely similar and yield informative negatives. To make this deployable at industrial scale, we realize the technique in a real-time, end-to-end framework that maintains a live in-memory item pool and draws cluster-based negatives from it on the fly via global out-of-batch sampling (GOOBS). The framework integrates directly into production two-tower training and serving and scales to billions of training examples with minimal computational overhead. Experiments on four public datasets and a 14-day online A/B test in a large-scale production system show that the proposed technique outperforms widely used industry methods, while also helping to break recommendation feedback loops and substantially reducing popularity bias.