Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection

In three linesSafety Reflection Pretraining inserts short safety reflections into pretraining corpora to establish self-monitoring directly in language modeling. On 1.7B models pretrained on FineWeb-Edu, the method improves safety classification accuracy and substantially reduces success rates of inference-stage and finetuning attacks.

Summary generated by Claude — human-verified