Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

In three linesNew MoLF (Mixture of LoRA and Full) method for LLM adaptation that dynamically routes gradients between full fine-tuning and LoRA at optimizer level. Tested on Gemma-3-1B, Qwen2.5-1.5B/3B across SQL, Medical QA, and counterfactual knowledge tasks. MoLF-Efficient outperforms adaptive LoRA approaches by 20% (Fact) and 9% (Med/SQL). Code open-sourced.

Summary generated by Claude — human-verified