Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction

In three linesAsynchronous RL pipelines for LLM agents lose historical old logits required for PPO off-policy correction, entangling discrepancy repair with staleness correction. The paper proposes three acquisition strategies (snapshot, dedicated model, interruption) and a revised PPO-EWMA method to preserve decoupled correction semantics.

Summary generated by Claude — human-verified