Hidden Consensus:Preference-Validity Compression in Human Feedback

In three linesStudy on human feedback aggregation in RLHF: reducing heterogeneous judgments to a single scalar reward masks legitimate cultural, linguistic, or normative disagreements. Analysis of 321 preference events in Malaysia: 79% of prompts contain multiple acceptable responses that majority aggregation discards. Proposes approach preserving plurality of valid interpretations.

Summary generated by Claude — human-verified