Starting from standard GRPO, we apply off-policy correction consisting of masked importance sampling, score centering, and top-p sampling with mask replay. As a result, training on the mixed samplers becomes stable, as evaluated on SWE-bench Pro V2.
Modal describes off-policy GRPO corrections that stabilize SWE-bench Pro V2 training
AISummary
Modal says it stabilized GRPO training on mixed samplers by adding masked importance sampling, score centering, and top-p sampling with mask replay. It reports that the approach yields stable training as evaluated on SWE-bench Pro V2.
Post on XView on X
Source: Modal · x.comPublished · added here