Several new papers address fundamental challenges in training and deploying large language models, with a focus on credit assignment and multi-agent reasoning.
According to arxiv.org, researchers introduced Policy Aligned Critic Training (PACT), which addresses token-level credit assignment in reinforcement learning for LLM post-training. The method formulates three regularity conditions—Completeness, Prefix Consistency, and Neutrality—that “uniquely determine token-level credit.” PACT adopts an Actor-then-Critic update order to better align the critic with updated policies. In mathematical reasoning benchmarks, PACT achieved 72.87% average accuracy across four benchmarks, outperforming GRPO and PPO by 8.80 and 13.16 percentage points respectively. On SWE-bench Verified, PACT achieved 67.4% pass rate, exceeding PPO, GRPO, and SAO by 2.4, 2.0, and 3.8 percentage points.
Separately, arxiv.org published research on VERPO (Verified Evidence Regularized Policy Optimization), which “converts evidence-conditioned guidance into reward-aligned token-level credit assignment.” The method decomposes teacher guidance into evidence-free reference terms and signed, evidence-induced corrections at each token.
Meanwhile, research on multi-agent systems revealed potential vulnerabilities. According to arxiv.org, a study evaluating 22,500 deterministic trajectories across three datasets demonstrated that “simulated social pressure triggers an algorithmic ‘Bystander Effect,’ inducing severe cognitive loafing.” The research uncovered a “Sovereignty Gap” where models “frequently compute the correct derivation internally but suffer ‘Alignment Hallucinations’—actively subjugating empirical evidence to sycophantically appease a simulated swarm.”