SuperThoughts: Reasoning Tokens in Superposition

In three linesSuperThoughts compresses consecutive CoT token pairs into single latent representations and decodes two tokens per step via a lightweight Multi-Token Prediction module. Tested on Qwen2.5-Math (1.5B, 7B, 14B), the approach reduces CoT length by 20-30% while maintaining accuracy (1-2 point degradation on MATH500, AMC, OlympiadBench, GPQA-Diamond).

Summary generated by Claude — human-verified