Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap

In three linesNovel data selection strategy for LLM alignment based on DPO implicit reward gap. Method selects harder examples (smaller reward gaps) and achieves superior performance with only 10% of original data across multiple benchmarks.

Summary generated by Claude — human-verified