BAGEL preprint reports prompt defenses built from small classifiers

Researchers report that BAGEL, an ensemble of small classifiers that detects malicious prompts, outperformed popular moderation APIs and most tested white-box defenses in a single-turn evaluation. The preprint reports an F1 score of 0.757 with an ensemble equivalent to 430M parameters, or 0.772 with a learned router and 324M parameters. BAGEL combines predictions from selected classifiers trained on different attack datasets. New attacks can be addressed by training and adding one 86M-parameter classifier rather than retraining the whole ensemble. Evaluation used held-out datasets excluded from training and calibration. Its multi-turn F1 score was 0.660, below its single-turn results but above the billion-parameter methods tested.