Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap

In three linesNovel data selection strategy for LLM alignment based on DPO implicit reward gap. By targeting harder preference examples (smaller gap), the method achieves superior performance with only 10% of original data across multiple benchmarks.

Summary generated by Claude — human-verified