Approximation in Contrastive Representation Learning
The research investigates how well standard inner-product scoring functions approximate Contrastive Representation Learning's true objective, Pointwise Mutual Information (PMI). The study proves that while finite dimensions introduce error, this error can be controlled. It shows that for structured data, better approximation rates are achieved and provides a fast calibration rate for empirical performance.