The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment

In three linesStudy on LLM overgeneralization beyond training data. Authors propose the Piggyback Hypothesis: chat-template tokens propagate finetuned behaviors to out-of-distribution domains. They introduce Token-Regularized Finetuning (TReFT) to mitigate emergent misalignment, achieving 33.5% more reduction than data interleaving on Llama-3.1-8B legal domain finetuning.

Summary generated by Claude — human-verified