TRACE preprint reconstructs removed training text from model snapshots

Researchers report that TRACE can reconstruct training text targeted by approximate unlearning, a process intended to remove information without fully retraining a language model. The attack requires access to model snapshots from before and after unlearning, but no prior knowledge specific to the removed data. TRACE compares changes in embedding parameters to select candidate tokens, then scores possible sequences using differences between the models. The preprint reports high-fidelity reconstruction across multiple models, datasets and six unlearning methods. Its findings make access to retained snapshots a specific privacy concern for organisations keeping them for auditing or rollback.