# Preprint compresses language-model weights into 16-64 shared values

_Published Monday, October 12, 2026 at 8:16 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report in a preprint that replacing each language-model weight matrix with 16-64 shared values preserves strong accuracy without retraining in Llama 3.1-8B-Instruct and SmolLM2-135M. The method groups numerical weights into clusters, providing a way to compress models on disk. Fine-tuning only the shared values recovers 30-40% of the remaining accuracy loss.

Experiments across four models from two architecture families suggest that preserving the ordering of stronger and weaker connections matters more than retaining exact values. Ordering alone is insufficient: performance also depends on weight variance and the positions assigned to clusters, while early layers are more sensitive to changes.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2603.17917)

---
Canonical: https://techandbusiness.org/newswire/Qxyqp1M3Icw5p3CV9AW7s9
Published: 2026-10-12T12:16:22.593Z
Story chronology: 2026-10-12T04:00:00.000Z
Retrieved: 2026-10-12T14:09:37.695Z
Publisher: Tech & Business (techandbusiness.org)