Quantifying Local and Global Fidelity of Nonlinear Embeddings

arXiv:2611.00290 · cs.AI · Submitted 2026-10-30 · Read on arXiv

cs.AI

Submitted: 2026-10-30

Updated: 2026-11-03

Comments: 29 pages, 17 figures, 12 tables

Code: https://github.com/hank08819/LG2Test

License: http://creativecommons.org/licenses/by/4.0/

The gist: Dimension reduction rests on a claim: t-SNE and UMAP preserve local structure, PCA global structure.

Terminology

Abstract

Dimension reduction rests on a claim: t-SNE and UMAP preserve local structure, PCA global structure. Our DLP curve is the share of each point's k nearest neighbors a map keeps. Corrected for chance, it gives L at k=10 and G over 30 to 50% of the data, local and global fidelity. At the midpoint the curve is provably a rank correlation of distances. The LG reversal test decides, with a stated error rate, which of two maps is local and which global. On twelve datasets of up to 1.38 million points, G agrees with established global scores at 0.84 and 0.93, and the area under R NX at 0.22 and 0.30. The entity fidelity E, read at each point's entity size, predicts which entities a map loses beyond L and G where entity sizes differ. In an agent memory of one million passages, t-SNE and PCA cross at 150,000 passages (p=2.6e-5).

Sources

Related papers