When Softmax Fails at the Top: Extreme Value Corrections for InfoNCE
Signal
72
Hype
15
In three linesInfoNCE, the standard contrastive learning objective, relies on a statistical assumption misaligned with modern normalized embeddings. Authors propose WEINCE, a simple modification using extreme value theory and anchor-wise online batch statistics to correct softmax logits without additional parameters. Consistent improvements across 5 vision benchmarks.Read source
Your take?
Summary generated by Claude — human-verified