Strategic Over-Parameterization for Generalizable Low-Rank Adaptation

In three linesLoRA-Over improves parameter-efficient fine-tuning (PEFT) by enriching the optimization landscape during training via auxiliary over-parameterization, then collapsing this enrichment into standard LoRA structure at inference. Evaluated on GLUE, MT-Bench, GSM8K, and HumanEval with LLaMA 2-7B and 3.1-8B, the framework consistently outperforms vanilla LoRA with no additional inference cost.

Summary generated by Claude — human-verified