Preprint reports sentiment retrieval gains from small embedding heads

Researchers report in a preprint that a small trained component can improve sentiment retrieval from text embeddings, the numerical representations used to compare documents. Their Sentiment-Injected Sentence Embeddings method improves retrieval MAP, a ranking-quality measure, by up to +26.0 across tested encoders released since 2022. The method trains a 0.13M-0.69M-parameter component on a frozen encoder in seconds on a CPU, using labelled training data. With that component, a 0.6B encoder at 128 dimensions matches a 4B encoder given the same treatment, scoring 91.8 versus 90.3 MAP. The gains involve specialization: the compact component loses 38-41 Spearman points on the STS-B semantic similarity benchmark. A variant combined with the original embedding reduces that loss to 2.4-12.0 points.