The post explains a concrete method for faster weight sync in large-scale RL and its open-source release, useful for teams training across clusters.
Really nice work from @humansand.
Built on the delta weight sync in Miles, taken much further, and contributed back to Miles so everyone on the stack gets it.
This is exactly the kind of collaboration we hoped Miles would enable. Thanks to the humans& team!
Large-scale RL has a problem: weight updates from trainers to samplers are slow and expensive We’re sharing weight teleportation: faster weight updates than existing intra-cluster techniques with internet speeds, enabling samplers anywhere (maybe even space!) Weight teleportation is so efficient (<0.23s sampler pause time, ~3s non-blocking compression + prepare), the bottleneck ends up being the GPU hardware’s decompression ASIC We unlock these gains by using GPUs for massively parallel compression of model weight changes, engineering a heterogeneous decompression and patching pipeline, along with other optimizations and overlaps. These techniques are co-designed with hardware-native features in NVIDIA’s Blackwell generation of chips We’ve open-sourced weight teleportation, so everyone can use it today
RadixArk · x.comPublished · added here
Original title: Really nice work from @humansand.
