Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Reference.

arXiv:2605.14220v2 Announce Type: replace-cross Abstract: Modern LLM RL systems separate rollout generation from policy optimization. These two stages are expected to produce token probabilities that match exactly. However, implementation differences can make