Agent privacy benchmark finds local checks miss 46.9% of recovered exposure

Researchers introduced ASLEval, a privacy benchmark that tracks unauthorized disclosure across every declared visible exit in a tool-using AI agent session. In tests spanning several enterprise-style environments and independently implemented runtimes, checking only the expected outlet missed 46.9% of the exposure recovered by examining all visible exits. The preprint says attacker self-reports also produced omissions and many false discoveries. Restricting information returned to the model changed the exposure path but could prevent successful completion of legitimate tasks, making task utility a necessary part of privacy evaluation.