Preprint compresses language-model weights into 16-64 shared values

Researchers report in a preprint that replacing each language-model weight matrix with 16-64 shared values preserves strong accuracy without retraining in Llama 3.1-8B-Instruct and SmolLM2-135M. The method groups numerical weights into clusters, providing a way to compress models on disk. Fine-tuning only the shared values recovers 30-40% of the remaining accuracy loss. Experiments across four models from two architecture families suggest that preserving the ordering of stronger and weaker connections matters more than retaining exact values. Ordering alone is insufficient: performance also depends on weight variance and the positions assigned to clusters, while early layers are more sensitive to changes.