Researchers have introduced a new method for aligning AI models at inference time that avoids the computational costs and data requirements of traditional fine-tuning approaches, according to a paper published on arxiv.org.
According to the paper titled “Inference-Time Nash Alignment” (arXiv:2609.08082v2), existing preference-based fine-tuning methods like RLHF and DPO “require substantial compute and large preference datasets” and need direct access to model parameters that many state-of-the-art models don’t provide. The researchers formulated inference-time alignment as “obtaining a Nash equilibrium of a two-player zero-sum game between policies.”
The team proposed two algorithms: Best-of-Nash (BoN) and Nash Mirror Descent (NMD). According to the paper, both algorithms “achieve a duality gap that matches the problem lower bound.” Testing on three datasets showed the methods “substantially outperform the base policy, converging to the performance of the fine-tuned models,” with NMD remaining “robust across the regularization parameter.”
Separately, arxiv.org published research questioning alignment midtraining (AMT) effectiveness. The study (arXiv:2609.20412v1) tested models up to 110 billion parameters and found that “a tiny fraction of finetuning data which suggests a competing motivation erases the effects of AMT.” The researchers concluded there is not “sufficient public evidence” to confirm midtraining can address “core difficulties inherent in aligning powerful AI systems.”