Researchers at the University of Washington and NVIDIA report that separating what a robot did from how well it fulfilled an instruction can improve AI-generated rewards for robot learning. Their Describe & Assess method, or D&A, uses existing vision-language models without robotics-specific training of the reward model. The work was posted to arXiv on November 2, 2026; peer review has not been verified.
Reward models supply signals that help a robot learn which actions advance a task. The authors observed that direct video prompting can award progress to failed executions or reduce assessment to a binary success judgment. They describe this behavior as confirmation bias, although their engineering experiments do not establish a psychological cause for the errors.
D&A first extracts scene entities using the video and task instruction. A separate description stage then narrates the observed actions without receiving the instruction directly, although the entity list carries information selected using it. An assessor receives the description and instruction, defines observable conditions for success, and scores progress from 0 to 1 against those conditions. Scores for sampled frames are interpolated across the trajectory. This anchors evaluation to a stated goal rather than assuming that later frames necessarily show greater progress.
On the authors’ D&A-Bench, the Gemini 3 Flash version achieved a final-trajectory ranking correlation, Kendall τₐ, of 0.67. Reported scores were 0.56 for Robometer, 0.43 for RoboReward, 0.27 for GVL and 0.24 for TOPReward. D&A exceeded direct prompting by 0.23 correlation units. These measure agreement with trajectory ordering, not percentages of tasks completed. The study also evaluated external benchmarks RBM-EVAL-OOD and RoboRewardBenchmark, using ranking correlation and mean absolute error, respectively.
The practical tests covered three physical robot tasks. For online learning to put fruit in a bowl, the authors report success rising from 10% for the initial policy to 100% with D&A after approximately 90 minutes. TOPReward reached 5%, a difference of 95 percentage points from D&A. Evaluation counts and uncertainty estimates were not supplied for these results.
Offline experiments used 150 demonstrations for transferring a book from a shelf to a box, including 50 optimal, 50 suboptimal and 50 failed executions. The baseline succeeded 20% of the time. The paper describes D&A’s improvement as “+80%” without clearly distinguishing relative percent from percentage points. In a second book task, successful D&A rollouts took 18.8 ± 1.8 seconds, versus 28.5 ± 9.2 seconds for the baseline. Neither the meaning of ± nor the number of successful rollouts was specified; the timing comparison excludes failures.
The component tests show that separation alone is insufficient. Table I evaluates RBM-EVAL-OOD using Gemini 3.7 Flash and reports ranking correlations of 0.486 for direct prompting, 0.561 with explicit success criteria, and 0.626 for full D&A. Separating the stages without entity grounding or success criteria scored 0.343. The table implies gains of 0.075 and 0.140 over direct prompting for success criteria alone and full D&A; the accompanying prose gives conflicting gains.
The evidence remains limited to author-run evaluations, with no independent replication supplied. D&A-Bench contains 54 trajectories across 14 tasks; ten instruction variants per trajectory do not constitute additional independent robot executions. Its claimed separation from model training data is not independently established, and comparisons do not uniformly use the same base model. Principal results lack confidence intervals and independent training-run counts.
D&A also uses the complete video, including later observations, to assess earlier progress. That supports retrospective reward labeling but does not establish performance for streaming assessment without future frames. The findings support a promising reward-design approach in the tested settings, with broader reliability still unresolved.
Source: https://arxiv.org/abs/2611.03622
Cover illustration generated with AI; conceptual, not research evidence.