Preprint demonstrates attacks that replace what vision-language models perceive

Researchers report in a preprint that small image and video perturbations can make vision-language models identify a target object while denying the original object. Complete replacement reached 38% in image tests at ε = 4/255 and 35.9% in video tests at ε = 1/255. The attack aligns internal representations of the source and target images within the model. Success requires the model to name the target, confirm its presence and deny the source. The researchers also observed models weaving contradictory visual signals into a coherent narrative. The tests use a white-box threat model, requiring access to the model's internal workings.