On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning.

arXiv:2610.40945v1 Announce Type: cross Abstract: On-policy reinforcement learning for large language models assumes that training responses are sampled from the current policy. In practice, an inference engine samples the responses and a separate trainer comp