Constitutional AI and RLAIF: How Natural Language Principles and Automated Critiques Scale LLM Alignment
Early alignment frameworks for large language models relied almost entirely on Reinforcement Learning from Human Feedback (RLHF). While RLHF transformed raw base models into usable assistants, the approach faces structural scaling bottlenecks. Collecting tens of thousands of high-quality human preference annotations is slow, expensive, and logistically complex. Furthermore, human annotators frequently disagree on nuanced safety boundaries, suffer psychological fatigue when reviewing harmful outp
1 min