Preprint converts selected language-model attention layers with limited cache savings

Researchers have proposed a method for replacing selected attention layers in pretrained language models while rejecting conversions that fail fixed quality checks. In tests, the TinyCeNN-LM framework accepted three layers in SmolLM2-135M and three full-attention layers in Qwen3.5-0.8B. Its Integrated Memory implementation reduced total cache use by up to 6.01% while keeping perplexity changes between -0.07% and +0.93%. The study does not establish a general attention replacement or a speedup, and its downstream check covered only 200 items.