Steerable Cultural Preference Optimization of Reward Models

In three linesNovel SCPO algorithm for training reward models that balance diverse cultural preferences across subcommunities. Achieves 7-point improvements for minority reward models on PRISM and GlobalOpinionQA (7 countries), with 280% better training data efficiency than full-finetuning.

Summary generated by Claude — human-verified