Training examples encountered only once may help language models generate scientific targets they were not trained to reproduce, according to a preprint posted on arXiv on November 2, 2026. In controlled fine-tuning experiments, the authors report better exact recovery of held-out materials formulas and regulatory DNA sequences as these rare examples became more prevalent. Peer review has not been verified.
The study tested gpt-oss-20b and gemma-3-12b-it, varying the proportion of operationally defined “monofacts” from 0% to 100% in 10-percentage-point steps. For materials, a monofact was a chemical system—a set of constituent elements—appearing once in the constructed training set. For genomics, it was a sequence with no other training example overlapping by more than 150 nucleobases. These definitions are task-specific proxies for the semantic facts used in the theory.
Materials training used a Materials Project snapshot dated November 14, 2024; evaluation targeted formulas first appearing in the September 25, 2025 snapshot. The genomics task used a random holdout from a dataset released in October 2024, after both models’ reported knowledge cutoffs. Models proposed modifications to 200-nucleotide DNA sequences targeting regulatory activity across K562, HepG2 and SK-N-SH cells. These arrangements reduce some contamination concerns but do not independently establish that evaluation information was absent from all training inputs.
At each monofact rate, the first training stage used three independently sampled datasets, each containing 10,000 materials examples or 20,000 genomics examples. A second stage used a separate 4,000-example dataset and reasoning traces attributed to Claude Opus 5. Success meant exactly matching a held-out target, either with one deterministic output or among 16 stochastic generations.
The authors report that recovery increased monotonically with monofact rate across both domains, model families, training stages and evaluation settings. Generating 16 candidates and adding reasoning supervision improved absolute recovery while preserving the trend. The reported gpt-oss-20b analysis also retained the relationship under low, medium and high reasoning effort. Numerical recovery rates and effect sizes are unavailable in the supplied text.
The theoretical explanation draws on “missing probability mass”: the probability associated with outcomes absent from a training sample. Singleton observations help estimate that mass. Under assumptions about calibration and a structured frontier of plausible candidates, the authors derive bounds linking monofact rate to expected discovery. More monofacts alone do not guarantee success; models must also direct unseen outputs toward plausible candidates. In a held-out materials test drawn from the training-period snapshot, higher monofact rates reportedly increased both recovery and outputs classified as hallucinations.
Here, “discovery” means recovering existing held-out targets. The experiments do not demonstrate prospective laboratory discoveries, material stability or experimentally verified function for newly generated DNA candidates. Evaluation-set sizes, numerical confidence intervals and significance tests are unavailable in the supplied evidence. The results support a role for training-data construction within these tested pipelines, while leaving open whether the relationship generalizes to broader scientific discovery.
Cover illustration generated with AI; conceptual, not research evidence.