New Framework Improves LLM Self-Correction by Redirecting Reasoning Mid-Stream

New Framework Improves LLM Self-Correction by Redirecting Reasoning Mid-Stream

Researchers have developed a novel approach to help large language models correct their reasoning errors by learning where to intervene during the generation process, according to a paper published on arxiv.org.

According to the arxiv.org paper, large language models struggle to reliably identify and correct errors in their own reasoning. The new framework, called UPTURN (User-Prompt TURN injection at a learned position), addresses this by training a small 130K-parameter MLP head on a frozen encoder using only final-answer correctness to determine optimal intervention points.

Unlike previous approaches that fine-tune LLMs to generate revisions or feedback, UPTURN retains a selected prefix of the model’s output, injects an instruction as a new user turn, and regenerates the suffix. According to the research, this approach requires no LLM fine-tuning.

The researchers found that “under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model’s course,” as stated in the paper.

According to the results, UPTURN improved average accuracy across four reasoners and six math benchmarks in every setting over a 32-rollout baseline. The framework also outperformed self-correction and reflection-token injection baselines under matched correction budgets, and transferred across reasoners and to GPQA-Diamond without retraining.