Investigation into In-Context Learning Capabilities of Transformers

In three linesSystematic empirical study of in-context learning capabilities in transformers on Gaussian-mixture binary classification tasks. Authors analyze how test accuracy depends on input dimension, number of in-context examples, and pre-training task diversity. They characterize benign overfitting emergence and identify critical parameter regions for ICL success.

Summary generated by Claude — human-verified