Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the different importance sampling based train-inference mismatch methods in RL. These have been the biggest focus of RL algorithm research since the GRPO takeoff, imo.
Lambert highlights importance sampling methods for RL train-inference mismatch
AINathan Lambert says he was reading a score-centering RL paper for notes in his book and found a table in its appendix comparing importance-sampling methods for train-inference mismatch in RL. He argues these methods have been the main focus of RL algorithm research since the GRPO takeoff.
Source details
Nathan Lambert · x.comPublished · added here
Original title: Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the differen...