The Long-Term Effects of Data Selection in LLM Fine-Tuning

In three linesStudy on long-term effects of data selection during multi-stage LLM fine-tuning. Authors show that short-term optimal strategies (loss-based, gradient-based, diversity-based) can slow future learning and increase catastrophic forgetting. They propose LHAS (Long-Horizon Aware Selection) to evaluate selection as a global training intervention.

Summary generated by Claude — human-verified