arXiv survey maps parameter-efficient continual fine-tuning for large models

A survey posted on arXiv reviews Parameter-Efficient Continual Fine-Tuning, the combination of continual learning methods with parameter-efficient fine-tuning for large pre-trained networks. The authors argue that large pre-trained models still inherit a strong dependence on the i.i.d. assumption, which limits adaptation when new data and tasks arrive sequentially. Continual learning targets that lifelong setting, while parameter-efficient fine-tuning methods adapt models through small, efficient modifications that can approach full fine-tuning performance on a given scenario. According to the abstract, those PEFT techniques still struggle when the model must adjust across multiple tasks over time because of catastrophic forgetting. The survey first overviews continual learning algorithms and PEFT methods, then reviews the state of the art on Parameter-Efficient Continual Fine-Tuning, including approaches, evaluation metrics, and possible research directions. The paper frames the goal as highlighting the synergy between continual learning and parameter-efficient fine-tuning and guiding further work in the area.