>/BASH

HARP preprint reports recommendation gains from profile-guided retrieval

HARP improved offline ranking on four benchmarks using aligned user and item profiles with a frozen LLM. Component tests point to profile-guided retrieval as the main contributor, but imported baseline scores and unverified leakage safeguards limit the conclusions.

HARP preprint reports recommendation gains from profile-guided retrieval

A recommendation system can keep the language model fixed while changing which evidence it retrieves. That is the approach behind HARP, whose authors report higher offline ranking accuracy on four public benchmarks. The research, by Zhen Tao and colleagues, was posted to arXiv on November 3, 2026. It is a preprint; peer review has not been verified, and no independent replication is supplied.

HARP addresses a mismatch in memory-based recommenders: users may have structured descriptions, while items lack comparable profiles, and profile information enters only the final ranking prompt. The proposed system aligns user and item profiles across four dimensions—preferences, quality, context and audience—and uses their compatibility to reweight collaborative neighbours before their memories reach the ranking model. This moves some profile processing outside the prompt’s limited space.

User profiles are generated by an LLM from interaction histories. Item profiles are extracted through deterministic rules applied to metadata and reviews, requiring no additional LLM calls. A reflective step uses the interaction preceding the held-out test item to propose a profile revision; an optional rule-based check can reject it. Training-free here means no gradient updates: profiling and reflection still require computation and LLM calls.

The evaluation uses a frozen gpt-4o-mini backbone and the original InstructRec splits for Books, Goodreads, MovieTV and Yelp. Main experiments retrieve 16 collaborative neighbours, chosen from 4, 8 and 16 on a development split, and use one warmup interaction and one reflection pass per user. The retrieval count is separate from the ranking cutoffs. Hit@1 measures whether the held-out target ranks first; the study also reports Hit@3/5 and normalized discounted cumulative gain at 3/5.

Against MemRec scores imported from its publication, reported Hit@1 rises from 0.512 to 0.547 on Books, 0.400 to 0.469 on Goodreads, 0.588 to 0.645 on MovieTV, and 0.487 to 0.604 on Yelp. These are increases of 3.5, 6.9, 5.7 and 11.7 percentage points. The authors report relative gains of 6.9%, 17.2%, 9.7% and 24.0%, respectively. HARP leads all five ranking metrics across the four datasets in the supplied main table.

Component tests provide more direct evidence about the mechanism. On a 1,000-user Yelp subset with a shared backbone, memory alone reaches Hit@1 of 0.580, profiles with retrieval gating reach 0.635, and full HARP reaches 0.643. Profiles used in the prompt without retrieval gating score approximately 0.578–0.580. This supports retrieval gating as the largest contributor in that configuration. The final step combines prompt injection and reflection, so its extra 0.8 percentage points cannot be assigned to reflection alone.

The authors report Holm-corrected one-sided binomial p-values below 10^-9 for all four main Hit@1 comparisons. Those tests use published baseline rates rather than paired baseline predictions, treat those rates as fixed, and omit their uncertainty. Imported results also do not establish identical model versions, prompts, computing budgets or execution conditions. Independence of user cases is not established because users share items and collaborative graph information.

Further limits remain unresolved in the supplied material. Exact test denominators, candidate construction and referenced methodological appendices are unavailable for assessment. It is unclear whether profiles, reviews and graph construction consistently exclude held-out or future information, so leakage safeguards cannot be verified. Latency and total cost are not quantified here. The findings support an offline engineering result on these four benchmarks, without establishing better user satisfaction, sustained adaptation or general superiority across models and deployed recommenders.

Cover illustration generated with AI; conceptual, not research evidence.

Original sources

View this story on Instagram