Cosmos preprint demonstrates real-time interactive world simulation

Researchers report in a preprint that they adapted Cosmos world foundation models to generate visual observations in real time as users or embodied agents take actions in virtual environments. They have made code and model checkpoints available for the framework. The system can predict the visual consequences of actions or generate observations from physical state maintained by an external simulator. Demonstrations cover camera navigation, robot manipulation, controllable video transfer and autonomous vehicle simulation with multiple views. The work supports virtual environments for training and evaluation; the reported demonstrations concern simulation rather than physical deployment.