Preprint reports accuracy gains from new language model compression methods

Researchers report that a new method for allocating compression across language model layers improves zero-shot accuracy by up to 4.8 percentage points over existing search strategies. The preprint tests Mistral, Qwen3 and Llama 3 families, with gains extending to models with 70B parameters. The method, Layer-wise Error Modeling Search, estimates each layer's importance and how errors propagate to decide where to retain more model capacity. A second method, KFAC-SVD, uses token-wise statistics to address rank deficiency, a problem in earlier compression approaches, and reports an average perplexity improvement of 15%. The results concern compressed model quality rather than demonstrated deployment savings.