Johns Hopkins paper links once-seen training facts to LLM scientific discovery

A Johns Hopkins University paper finds that language models make more scientific discoveries as the share of training facts seen only once rises from 0% to 100%.

The authors fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, and the trend held in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.