Researchers at the University of Texas at Austin have developed GUILD, a system that uses feedback about training data to decide which robot demonstrations to collect next. In an arXiv preprint posted November 3, 2026, the authors report improved target-task performance under fixed demonstration budgets in simulation and real-robot experiments. Peer review and independent replication have not been verified.
Robots trained through imitation learning learn actions from demonstrations, often collected through costly human teleoperation. GUILD addresses situations where most collection must happen in accessible auxiliary environments, while only a small demonstration dataset is available from the intended deployment setting. Its aim is to prioritize useful demonstrations before collection costs are incurred, rather than filter a large dataset afterward.
A vision-language model first proposes collection tasks, including environments, object arrangements and goals. GUILD then uses MAGIC, a metagradient attribution method, to estimate how demonstrations influence imitation loss on the target examples. It aggregates these estimates by task, collects additional demonstrations from tasks predicted to be most useful, and recalculates their utility. Rankings also inform later task proposals. The signal is a local estimate based on an offline loss proxy, rather than a direct measurement of how each new demonstration changes task success.
The researchers trained pretrained Octo policies using a 50/50 sampling mixture of collected demonstrations and target-task examples. Simulation experiments covered three target tasks drawn from 17 LIBERO-90 environments, with scripted demonstrations collected in the other 16 environments for each target. The authors report that GUILD matched or exceeded the strongest baseline at every evaluated collection budget. Comparators included random collection, uniform collection from model-proposed tasks, STRAP and Factored Scaling Curves. Exact success rates and most effect sizes are unavailable in the supplied figure text.
For real robots, a fixed-pool experiment selected demonstrations from 49 previously recorded tasks across six environments: two kitchens, two bathrooms and two offices. Across four target tasks, the authors report the highest success under equal collection budgets, also exceeding an All-Data baseline that included recordings from the target environment. This experiment sampled existing recordings; it did not directly measure prospective teleoperation savings. In a separate open-world setting allowing new task specifications, GUILD reportedly beat uniform collection from model proposals on every evaluated task, with approximately 25% higher average performance. The text does not clarify whether that figure means relative improvement or percentage points.
GUILD also reportedly exceeded average human-guided performance in comparisons involving eight roboticists in simulation and two experts in real-world collection. These small comparisons do not establish general superiority over expert collectors. Simulation results used three independent collection seeds, with five policies per evaluation and 100 rollouts per policy; reported means and standard errors were calculated across collection seeds. Real-world policies were evaluated over 20 rollouts, but independent collection repetitions are not specified.
The findings support targeted data collection within these limited settings. Target demonstrations served both attribution and policy training, so generalization to unseen target domains is not established. Frequent utility updates increase computation. Equal demonstration counts also do not establish equal collection time, setup effort or total cost. Statistical significance across the many comparisons is not documented in the supplied text.
Source: GUILD: Guided Iterative Data Collection for Imitation Learning, arXiv:2611.05368, version 1.
Cover illustration generated with AI; conceptual, not research evidence.