New Meta paper finds that a model distilled from watermarked API outputs keeps a hidden, key-specific trace in its weights, and that trace can be read from the copy's text alone.
Today's detector counts watermark tokens in the suspect's text and misses most copies made with low-temperature sampling.
They found that watermarked traces push a student's weights in a faint, key-specific direction. WM-Basin turns the suspect's API outputs into gradients of a fixed probe model and tests if they match the real key over decoys.
Across 1,200 distilled models at a 0.1% false-positive rate, it caught 79% with n-gram=1 watermarks, against 29% for token counting. Quantization, light pruning and RL post-training did not remove the trace, though more fine-tuning on clean data weakened it.
– arxiv. org/abs/2610.37266
Title: "Watermark Basins: Black-Box Detection of Distilled LLMs"