A coding agent that builds simulated tasks, paired with a video model that supplies realistic imagery, could reduce the physical demonstrations needed to train robots. Researchers affiliated with Peking University and NVIDIA report that their PyRUA-DataFarm pipeline produced training data supporting strong real-robot performance on five tasks. The work was posted to arXiv on November 3, 2026; peer review is not verified, and no external replication is supplied.
Human teleoperation records actions and images directly from a robot, but each demonstration requires physical collection. DataFarm instead starts with a task description and one real calibration recording from a head camera and two wrist cameras. A coding agent reconstructs the scene and cameras, develops a controller and establishes success checks in simulation. The calibration recording grounds the scene’s geometry and appearance; its actions are not reused as training labels.
The simulator generates trajectories and action labels, while a video model renders those trajectories using real reference images. The default renderer is a self-hosted MiniMax-H3 deployment, with all camera views generated together to encourage consistency. Separate agent roles, hidden success checks and automated video checks control admission to the dataset. These reviewers are internal components of the pipeline, rather than external evaluators, and their residual error rates are not established.
The team tested a dual-arm PiPER robot on nesting three cups, placing an apple on a scale, striking a drum at least twice, placing a block in a drawer and closing it, and folding a towel twice. For each task, π0.5 policies were trained on 100 episodes from each of three sources: DataFarm, real teleoperation and simulator rendering with domain randomization. Training settings were held constant. Each policy then underwent 20 physical trials per task in each condition, with randomized object placements shared across methods.
Across the five tasks in the original scene, DataFarm-trained policies recorded 93 successes in 100 trials, compared with 84 for teleoperation and 26 for simulator domain randomization. With unseen tablecloths and changing coloured lighting, the totals were 87, 46 and 21, respectively. DataFarm’s lead over teleoperation was therefore 9 percentage points in the original scene and 41 points under changed appearance.
Those results compare complete training pipelines. DataFarm’s training videos included variations in surfaces and lighting, while the teleoperation dataset showed the original scene. The robustness advantage therefore cannot be attributed to agent automation or rendering alone. The simulator baseline shared DataFarm’s trajectories, supporting the value of its observation-generation pipeline without separating the effects of real-image conditioning, video generation and appearance augmentation.
The authors report mean setup costs of $20.92 for task adaptation and $5.13 for a rendering specification, followed by $0.80 per accepted episode. Including setup, a batch of 100 episodes averaged $1.06 per episode. One hour of synchronized three-view data averaged $107 per task, compared with a $117 Chinese teleoperation quote and a $290 North American labour-only estimate. These comparisons use different accounting bases. Calibration collection, infrastructure and policy-training costs are not clearly covered, the low rendering costs depend on self-hosting, and the default and costed runs used different render-selection recipes.
The evidence remains limited to five selected tasks on one robot platform. The supplied evaluation reports no confidence intervals, statistical tests or repeated policy-training runs; its trials are nested within tasks and trained policies, rather than independent repetitions of the whole pipeline. The counts therefore do not establish equivalence or universal superiority. Performance also depends on adequate simulation: randomizing uncertain mass or friction may be insufficient for tasks requiring precise dynamics, and generated videos can retain defects despite review. DataFarm offers a promising route to cheaper training data, but broader testing and real-robot evaluation remain necessary.
Cover illustration generated with AI; conceptual, not research evidence.