RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training

In three linesarXiv paper showing RL applied directly to pre-training checkpoints (without prior SFT) is effective from early stages. Pre-training data composition impacts RL effectiveness more than model scale. Merging RL and SFT objectives via parallel averaging outperforms standard pipelines while preserving general capabilities.

Summary generated by Claude — human-verified