Robotics preprint reports 200-rollout evaluations costing approximately $3.50

Researchers report a reusable system for evaluating robotic manipulation at a scale of 200 rollouts for approximately $3.50 USD per replication in a new preprint. The system runs on serverless commodity GPUs, taking 38 minutes and 4.4 L4-GPU-hours across eight parallel shards. Their experiments also found zero genuine successes across 780 rollouts from three checkpoints trained to predict action sequences, while three single-step checkpoints achieved success rates from 2.6% to 11.8%. A control that replanned at every step partially linked the failure to the training formulation and multi-step prediction, rather than execution of action sequences without replanning.