Fine-Tuning Dynamics of In-Context Factual Recall in Transformers

In three linesTheoretical study of in-context learning dynamics in transformers. Authors formalize the IC-recall task where the model infers a hidden relation from examples and retrieves factual knowledge stored in parameters. Proof that fine-tuning converges to a specific attention pattern using polylogarithmic sample complexity.

Summary generated by Claude — human-verified