Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

In three linesFoLoRA, a forgetting-aware optimization framework, adapts foundation models via generalized Rayleigh quotient. It balances target task performance and preservation of pretraining capabilities by penalizing adaptation directions with low utility relative to forgetting. Tested on math, code, and instruction-following adaptation.

Summary generated by Claude — human-verified