SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding

In three linesSENSE improves retrieval-based speculative decoding by anchoring retrieval on target model hidden states for robust semantic alignment. A soft-gated evaluation module validates semantic equivalence rather than surface forms. On LLaMA and Qwen, SENSE achieves 4.09 mean acceptance length and 3.26x speedup.

Summary generated by Claude — human-verified