Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance

In three linesStudy of 25 embedding models on 5 MTEB tasks showing that nearest-neighbor overlap and magnitude differences in ICA strongly correlate (up to 0.97) with performance. Embedding tasks display varying degrees of linearity and reliance on local information retention.

Summary generated by Claude — human-verified