>/BASH

Pooling-aware training preserves more search quality with fewer document vectors

A Hugging Face experiment reports that training with compressed document representations improved retrieval at the same vector budget. Factor-4 pooling retained 97.8% of the trained model’s uncompressed benchmark score, but the comparison used only one training run per condition.

Pooling-aware training preserves more search quality with fewer document vectors

Training a search model to work with compressed document representations can help preserve retrieval quality as vector counts fall, according to an experiment Tom Aarsen reported on the Hugging Face blog on November 11, 2026. At approximately one quarter of the original document-vector count, the pooling-aware model scored 51.23 on NanoBEIR@13, compared with 49.03 for an ordinarily trained model.

Multi-vector retrieval models represent documents with multiple contextual token vectors. MaxSim scoring lets each query vector find its strongest document match, then adds those scores. This supports matching different query concepts to different document passages, but storing many vectors per document can make indexes large.

The experiment used Ward hierarchical clustering to merge similar document vectors. Each resulting group was represented by its mean, then normalized again; query vectors remained unchanged. Pooling-aware training gave equal weight to retrieval losses from full document representations and representations pooled by a factor of four. A straight-through estimator passed gradients back to the constituent token vectors without differentiating the discrete cluster assignments.

Aarsen trained two ModernBERT-base models with fresh 128-dimensional token projections for one epoch on the same 120,000 MS MARCO triplets. Both used seed 42, matching initialization and optimization settings, a training token cap of 300, and batches of 256 with cached mini-batches of 64. Short queries were expanded to at least 32 token positions. Only the training loss differed; no hyperparameter search was reported.

Evaluation measured mean nDCG@10, expressed as a percentage, on NanoBEIR@13. This metric measures ranking quality among the top 10 results. Factor-4 pooling reduced mean document vectors from 193.22 to 48.68, a 74.8% reduction. At that budget, pooling-aware training improved the score by 2.20 percentage points. Retention relative to each model’s own unpooled score rose from 94.6% to 97.8%; this ratio is distinct from the absolute improvement over the baseline. Unpooled scores also increased, from 51.84 to 52.39.

The pooling-aware model scored higher across all reported compression settings, including fixed caps and adaptive thresholds absent from its training objective. With adaptive pooling averaging 29.58 vectors per document, scores were 48.38 for ordinary training and 50.96 for pooling-aware training. The latter retained 97.3% of its unpooled score with 84.7% fewer vectors.

The evidence remains limited to one training run per condition, with no reported confidence intervals or seed variability. All compression comparisons reuse the same model pair. Aggregate results do not supply per-corpus scores, query counts, relevance-assessment details or training–evaluation overlap checks. Adaptive thresholds were calibrated separately using each encoder’s document embeddings from each evaluation corpus, with no reported relevance-label use; transfer to an unseen corpus was not established.

Vector counts are a storage proxy: total index size, end-to-end retrieval latency and deployment costs were not measured in the supplied report. Code and checkpoint links accompany the post, but independent reproducibility was not assessed. These are author-reported blog results; the supplied source establishes neither peer review nor independent replication.

Cover illustration generated with AI; conceptual, not research evidence.

Original sources

View this story on Instagram