MIT robot memory system corrects uncertain object descriptions in real time

MIT researchers developed a robot memory system that corrects uncertain object descriptions in real time while a mobile robot moves. The researchers report that UQ-DAAAM resolved ambiguities for about 92 percent of objects using two additional captions per object and increased description accuracy by 5 percent compared to the state of the art. The system compares descriptions of the same object from different viewpoints, selects images that reduce uncertainty, and combines captions into an updated memory entry. Its selection method avoids rerunning the vision-language model, reducing computation. Applying the system to mobile robots that also manipulate objects is a planned next step.