1. 08:00
   Apple Machine Learning Research

   Research reveals the theoretical limits of language model alignment

   Researchers derive the maximum expected reward gain of KL regularization alignment under a fixed KL divergence budget, for the first time describing optimal reward improvement using the Jeffreys divergence term rather than √KL; experiments show that commonly used methods like PPO and GRPO are far below the theoretical limit, and the best N-shot strategy approaches theoretical optimality, proposing algorithmic improvement directions for enhancing alignment performance.

Hover or tap the chart to see each hour; use the arrow keys on a keyboard.