Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment

In three linesNew arXiv paper proposing HRC (Hybrid Reward-Cyclic), a reward model explicitly decomposing human preferences into transitive (scalar) and cyclic (vector) components via game theory. Introduces DSPPO (Dynamic Self-Play Preference Optimization) for alignment. Results: +1.23% on RewardBench 2 vs GPM, 44.75% win-rate on AlpacaEval 2.0 with Gemma-2B-it.

Summary generated by Claude — human-verified