Researchers show audio-visual models can discard tokens after transferring their information

Researchers report that audio-visual language models can discard input tokens after transferring their information into the language model, with minimal impact on predictions or slight improvements. The arXiv study demonstrates the approach across multiple tasks and datasets using Qwen2.5-Omni and Video-SALMONN2 Plus at 3B and 7B scales. The researchers traced how sound and image information moves through the networks. Video inputs followed a sequential pathway, while multiple interleaved audio-visual items used parallel streams. The findings support more efficient inference, but establish results for the tested models and configurations.