DPO

Self-Rewarding Language Models: How Iterative DPO and LLM-as-a-Judge Form Autonomous Self-Alignment Loops

Standard post-training alignment pipelines rely on frozen reward models trained on static human feedback datasets. While Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) effectively steer model outputs toward human preferences, they face a fundamental scalability bottleneck: human annotators cannot evaluate superhuman reasoning or generate labels at the scale required for continuous self-improvement. Self-Rewarding Language Models, introduced by Meta AI