Skip to content
Trending storyDeveloping

TIM masking scheme switches from relative to absolute difference

1 article1 sourcesince Oct 19Last article 2h ago ·

Overview

AISummary of 1 article

@nrehiew_ proposes replacing the relative difference with the absolute difference in a new TIM masking scheme.

Prime Intellect says its IcePop method collapsed around step 1.2k, when trainer-inference KL rose 20×, while its IPO method trained through the same run. IPO masked about 0.25% of tokens in agentic SWE RL with GLM-4.5-Air, according to Prime Intellect.

Written by AI from the articles below · updated Oct 19, 5:19 PM ET

Check the sources:

Latest developmentOct 19, 5:16 PM ET
TIM masking scheme uses absolute difference instead of relative difference

Article timeline

The articles in this story. Times are ET.

Oct 19
  1. whX
    TIM masking scheme uses absolute difference instead of relative difference

    AI@nrehiew_ proposes using the absolute difference rather than the relative difference in a new TIM masking scheme. Prime Intellect says IcePop collapsed around step 1.2k when trainer-inference KL rose 20×, while IPO trained through, masking about 0.25% of tokens in agentic SWE RL with GLM-4.5-Air.

    Image from @nrehiew_'s post

Heat Trend

Not enough continuous observations to show a trend yet.