There is no need to train from scratch.
We convert existing pretrained models with just a few billion tokens of distillation while freezing the base model. For Qwen3.6-35B-A3B, ZHA adds only 81,920 trainable parameters (~0.00024% of the model).
Zyphra converts pretrained models with ZHA using 81,920 trainable parameters
AIZyphra says it converts existing pretrained models through distillation using only a few billion tokens while keeping the base model frozen, so no training from scratch is needed. For Qwen3.6-35B-A3B, its ZHA method adds only 81,920 trainable parameters, about 0.00024% of the model.
Source details
Zyphra · x.comPublished · added here
Original title: There is no need to train from scratch.