Architecture-Aware Reinforcement Learning Makes Sliding-Window Attention Competitive in Math Reasoning

In three linesSWARR combines sliding-window attention (SWA) with reinforcement learning for mathematical reasoning. After supervised conversion of a pretrained SA model, RL adapts self-generated trajectories to SWA constraints, narrowing the performance gap while preserving linear-complexity attention. Experiments on mathematical reasoning benchmarks.

Summary generated by Claude — human-verified