Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective

In three linesComparative study of fine-tuning vs. in-context learning on LLMs using formal language tasks. Fine-tuning outperforms ICL on in-distribution generalization, but both perform equally on out-of-distribution. Inductive biases diverge at higher proficiency levels. ICL shows sensitivity to vocabulary and model size.

Summary generated by Claude — human-verified