Direct Preference Optimization (DPO)

Aligning language models with human preferences without an explicit reward model — DPO derives a closed-form objective from RLHF's KL-constrained problem.