DEL: Digit Entropy Loss for Numerical Learning of Large Language Models

In three linesDEL (Digit Entropy Loss) is a novel loss function to improve numerical prediction in LLMs. Tested on CodeLlama, Mistral, DeepSeek, and Qwen-2.5 across 7 mathematical reasoning benchmarks, it outperforms existing methods (MLE, Number Token Loss) by optimizing digit entropy in a supervised manner and generalizing to floating-point numbers.

Summary generated by Claude — human-verified