This is cool as heck!
Imagine using a large LLM to directly edit a smaller model's weights and dramatically improve its capabilities.
It's like gene editing for neural networks. Even cooler than distillation!
Large LLM edits small model weights to boost math and instruction following
AIYuchen Jin says using a large LLM to directly edit a smaller model's weights could dramatically improve its capabilities, calling it gene editing for neural networks. The linked post reports that GPT-6 Astra changed only 0.016% of the weights in Qwen3.5-0.8B without gradients or training.
Yuchen JinVerified on X
@Yuchenj_UW
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://rikiyat.github.io/blog/weight-space-programming/