Researchers have published multiple approaches to improving how AI models learn from feedback, focusing on self-distillation and policy optimization techniques.
According to arxiv.org, a new framework called Visual-SDPO addresses how code-generating language models produce visual artifacts like charts and web pages. The paper states that “otherwise executable code often yields artifacts with visually salient defects, including overlapping elements, clipped text, broken alignment, low contrast, and overflow.” Visual-SDPO uses rendered visual feedback as “privileged context” and introduces Visual-Grounded Code Credit Weighting to trace defects back to responsible code statements. Testing on a Qwen3-VL-8B-Instruct model showed improvements of “more than 10 absolute points in the primary metric” over zero-shot baselines and “at least 2.4 points” over GRPO alone.
In related work, arxiv.org describes Group Relative Policy Optimization (GRPO) as a “baseline-free” alternative for neural combinatorial optimization that “normalizes advantages within groups of sampled trajectories.” The research found GRPO “avoids the training collapse observed with REINFORCE on TSP-100.”
Additionally, arxiv.org examined feedback alignment in self-distillation, finding that “step-aligned critique yields the largest gains, outperforming GRPO by 16.11 points.” According to the paper, this occurs because step-aligned feedback “targets only the tokens where reasoning fails, leaving correct behavior intact.”
Finally, arxiv.org introduced SocraticPO, which augments rollouts with “Socratic-style natural-language guidance” paired with reward decay to prevent shortcut learning.