# ReCoLoRA Method Recycles LoRA Adapters to Reduce Catastrophic Forgetting in Continual Fine-Tuning

_Friday, July 10, 2026 at 12:06 PM EDT · Science, AI · Latest · Tier 1 — Major_

A new method called ReCoLoRA addresses the problem of catastrophic forgetting when sequentially fine-tuning large language models with low-rank adapters.

Standard LoRA stacks new adapters on frozen weights, causing each task to overwrite the previous one. ReCoLoRA instead re-decomposes the current effective weight before each new task into a frozen residual, a slowly updated principal component, and a fresh adapter initialized from a randomized SVD of the pretrained weight with per-layer ranks chosen by an elbow criterion.

On a six-task continual GLUE sequence across four 7-8B backbones, ReCoLoRA achieved the best final average score on three of four models against rank-swept LoRA, PiSSA, AdaLoRA, and DoRA baselines while training fewer parameters. An oracle-routed task-bank variant provides an upper bound under full task isolation.

## Sources

- [arXiv](https://arxiv.org/abs/2607.07719)

---
Canonical: https://techandbusiness.org/newswire/bYgYz2WhulzV3IsdyJbVme
Retrieved: 2026-08-24T20:31:48.150Z
Publisher: Tech & Business (techandbusiness.org)