Risk-Aware Alignment Must Incorporate Reward Uncertainty.

arXiv:2410.23726v2 Announce Type: replace Abstract: We consider the problem of policy optimization with noisy rewards, where the policy is constrained to be in the neighborhood of a reference policy.We study the risk of the policy optimization process, defined