FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning

In three linesFuRA introduces full-rank parameter-efficient fine-tuning via spectral preconditioning through SVD decomposition. By freezing pretrained singular bases and optimizing only compact cores via block tensor-train factorization, FuRA outperforms full fine-tuning and LoRA on LLaMA-3-8B (+1.37 commonsense reasoning) and VLMs while maintaining LoRA-comparable efficiency.

Summary generated by Claude — human-verified