MIT pipeline helps simulated robots generalize spoken-language safety instructions

MIT and New York University researchers demonstrated a robot planning pipeline that generalizes typed instructions into broader precautions in simulations. Called FLEX, it turned a warning against spilling coffee on a laptop into a rule to avoid other electronics, including a nearby phone. The pipeline uses GPT-4o to identify broader preferences from specific corrections, then uses computer vision to remove objects no longer present from the robot's planning list. The three-step process takes about five seconds. Volunteers tested it with simulated robot arms serving coffee, moving a candle and carrying a bowling ball; the demonstrations did not involve physical robots.