Previously generated AI responses may help evaluators estimate benchmark scores even when models have answered different questions. In a preprint posted to arXiv on November 2, 2026, Hayden Helm and colleagues report improved predictions across two benchmark suites using Product Kernel Perspective Space, or PKPS. Peer review has not been verified, and no independent replication is supplied.
The approach addresses two problems: estimating a full-task score from a few answers, and predicting a model’s score on a task it has never answered using its responses elsewhere. Unlike methods restricted to identical questions answered by multiple models, PKPS weights comparisons between responses according to how similar their questions are.
The researchers embedded questions and responses with gemini-embedding-001 into 3,072-dimensional vectors, reduced them separately using principal component analysis, and constructed eight-dimensional representations of models. A five-nearest-neighbor regressor then predicted benchmark scores. Cross-validation selected query-kernel bandwidth and weights for ensembles combining PKPS with sampled scores or low-rank matrix completion, known as LRMC.
Experiments used selected dense subsets of HELM-Lite, with 93 models and 18 tasks, and Every Eval Ever, with 45 models and 16 tasks. The team deliberately masked observations to control coverage and question overlap. Target developers’ models were excluded from reference regression training. Results were measured as mean absolute error, or MAE, against full cached-evaluation scores and averaged over 16 subsampling seeds.
With one question per model–task pair and full task coverage on HELM, PKPS recorded MAE of 0.137, compared with 0.292 for the direct sample score. Combining the two reduced error to 0.126. On Every Eval Ever, one-question PKPS achieved 0.070, below both the one-question sample score’s 0.224 and the eight-question sample score’s 0.081. That specific comparison supports an eightfold reduction in query budget, rather than a general saving across all settings.
For entirely missing model–task scores, at 50% task coverage and 50% query depth, the PKPS–LRMC ensemble reduced MAE from 0.139 to 0.120 with 10 HELM models. With all 93, it improved from 0.105 to 0.100, but the per-task difference was not statistically significant. With all 45 Every Eval Ever models, error fell from 0.096 to 0.069, approximately 28% lower relative error.
The gains have limits. Direct sample scores eventually outperformed standalone representation methods as question budgets increased. The theoretical advantage over paired representations is asymptotic and conditional, including assumptions about exact reference scores, independent model sampling and, for stronger results, kernels and query distributions.
Controlled masking does not reproduce the selective missingness of real evaluation records. Shared models, tasks and possible model ancestry also create dependence; 16 subsampling seeds are not independent model cohorts. Performance on new tasks or different model populations remains uncertain. Embedding and computation costs were not quantified, so fewer queries do not establish lower total costs. The findings concern benchmark-score prediction within these evaluations, not broader model reliability.
Cover illustration generated with AI; conceptual, not research evidence.