>/BASH

Pyrite tests AI agents on a difficult protein–ligand pose-ranking benchmark

An arXiv preprint reports that AI agents outperformed five chemists on selected protein–ligand pose comparisons. Structured descriptors and AI-generated features helped, but the benchmark does not establish better drug discovery, and human comparisons depend on how abstentions are scored.

Pyrite tests AI agents on a difficult protein–ligand pose-ranking benchmark

Language-model agents can help distinguish between plausible predictions of how a small molecule binds to a protein, according to a study introducing the Pyrite benchmark. The strongest direct-reasoning configurations reached 67% accuracy on 484 evaluation pairs, against a 50% chance baseline. The work tests a computational selection task; it does not demonstrate improved medicines or experimental drug-discovery outcomes.

The paper, posted to arXiv on November 10, 2026, comes from researchers affiliated with Stanford University and Genesis Molecular AI. It is a preprint whose peer-review status is unverified, and no independent replication is supplied.

Modern cofolding systems predict protein and ligand structures together, potentially producing several plausible binding arrangements, or poses. Pyrite asks which of two predictions is closer to an experimental crystal structure. Its 1,500 pairs cover 285 structures and 1,799 unique predicted poses. A proprietary system generated candidates; the researchers retained the top 25 per query using the MM-GBSA energy score, then applied reference-based filters. Every retained pair contains a designated high-quality pose.

The authors supplied agents with PyriteRep, a representation containing 29 features describing molecular geometry, interactions and physicochemical scores. PyriteReasoner compares these descriptions through text reasoning. PyriteFeaturizer instead writes feature-extraction code for a logistic-regression classifier, optionally adding assessments from further language-model calls. Training used structures released before September 30, 2021; evaluation covered 90 later structures. Raw atomic coordinates alone generally produced near-chance results, while structured information helped.

PyriteReasoner’s best reported accuracy on all 484 evaluation pairs was 67%, with a reported 95% confidence interval of ±4 percentage points. On the human-annotated subset of 265 pairs from 56 structures, it reached 67% ±6 points. Five chemists, averaging more than 10 years of relevant experience, scored 45% ±4 points when abstentions counted as errors. Their accuracy among decisive responses was 57% ±5 points, making the size of the apparent human–AI gap sensitive to scoring rules.

Feature generation also showed promise. On that same subset, reinforcement learning using Group Relative Policy Optimization improved the code-only Qwen3.8-27B version of PyriteFeaturizer from 52% ±6 points to 69% ±6 points. The authors marked this 17-percentage-point gain statistically significant. Conventional logistic regression using existing quantitative features achieved 64% accuracy, showing that a simple statistical baseline was already competitive.

Choosing the pose with the lower MM-GBSA score achieved only 35% ±4 points across all evaluation pairs. That result applies to candidates already screened using the score. It does not establish that reversing the score would work generally.

The findings remain specific to a selected benchmark from one generation pipeline. They do not establish binding-affinity prediction, complete candidate-set ranking or prospective experimental success. Pairs can share structures or poses, and the supplied material does not establish whether uncertainty estimates account for those dependencies or multiple comparisons. The date split also does not by itself exclude related proteins or ligands across training and evaluation, or exposure during model pretraining. With only five experts and incomplete appendix-level verification, broader testing is needed to determine whether the gains transfer to other generators and practical workflows.

Cover illustration generated with AI; conceptual, not research evidence.

Original sources

View this story on Instagram