Skip to content
Trending storyDeveloping

Meta proposes WM-Basin to detect watermark traces in distilled LLMs

1 article1 sourcesince Oct 30Last article 2h ago ·

Overview

AISummary of 1 article

Meta researchers propose WM-Basin, a black-box method that detects distilled LLMs from their text outputs alone, without access to weights or logits.

Rohan Paul, summarizing the paper on X, reports that across 1,200 distilled models at a 0.1% false-positive rate, WM-Basin caught 79% of copies with n-gram=1 watermarks, against 29% for token counting.

The paper finds that a model distilled from watermarked API outputs keeps a hidden, key-specific trace that can be read from the copy's text. Per the report, quantization, light pruning and RL post-training did not remove the trace, though more fine-tuning on clean data weakened it.

Written by AI from the articles below · updated Oct 30, 4:24 AM ET

Check the sources:

Latest developmentOct 30, 4:05 AM ET
Meta paper proposes black-box detection of distilled LLMs via watermark basins

Article timeline

The articles in this story. Times are ET.

Oct 30
  1. Rohan PaulX
    Meta paper proposes black-box detection of distilled LLMs via watermark basins

    AIMeta researchers report a method called WM-Basin that detects distilled LLMs through their API outputs, without access to weights or logits. Across 1,200 distilled models at a 0.1% false-positive rate, it detected 79% of copies with n-gram=1 watermarks, versus 29% for token counting. The trace survived quantization, light pruning and RL post-training, though more fine-tuning on clean data weakened it.

    Image from @rohanpaul_ai's post

Heat Trend

Not enough continuous observations to show a trend yet.