# ReCoLoRA Method Recycles LoRA Adapters to Reduce Catastrophic Forgetting in Continual Fine-Tuning

_Friday, July 10, 2026 at 12:06 PM EDT · AI, Products · Latest · Tier 1 — Major_

![ReCoLoRA Method Recycles LoRA Adapters to Reduce Catastrophic Forgetting in Continual Fine-Tuning — Primary](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png)

A new method called ReCoLoRA addresses the problem of catastrophic forgetting when sequentially fine-tuning large language models with low-rank adapters. Standard LoRA stacks new adapters on frozen weights, causing each task to overwrite the previous one. ReCoLoRA instead re-decomposes the current effective weight before each new task into a frozen residual, a slowly updated principal component, and a fresh adapter initialized from a randomized SVD of the pretrained weight with per-layer ranks chosen by an elbow criterion. On a six-task continual GLUE sequence across four 7-8B backbones, ReCoLoRA achieved the best final average score on three of four models against rank-swept LoRA, PiSSA, AdaLoRA, and DoRA baselines while training fewer parameters. An oracle-routed task-bank variant provides an upper bound under full task isolation.

## Sources

- [arXiv](https://arxiv.org/abs/2607.07719)

---
Canonical: https://techandbusiness.org/newswire/bYgYz2WhulzV3IsdyJbVme
Retrieved: 2026-07-10T18:24:39.603Z
Publisher: Tech & Business (techandbusiness.org)