According to research published on arxiv.org, large language models can be manipulated to abandon correct beliefs through targeted persuasion, with a single argument collapsing model accuracy “to near zero, even when the argument is factually false.”
The study formalized this vulnerability as “adversarial persuasion” and introduced an adversarial reinforcement learning framework to train persuader agents. According to the paper, “optimizing persuasion strategies through trial and error exposes vulnerabilities that static prompting misses,” with RL-trained persuaders raising success rates from approximately 24% to over 93% against the training-time target model.
The research demonstrates that learned persuasion strategies transfer across models, achieving 83% attack success on Qwen-14B, 79% on Llama-3.1-8B, and 25% on GPT-4o-mini, according to arxiv.org. When researchers used a curriculum approach that “bootstraps on more persuadable open-weight models before targeting harder models,” GPT-4o-mini attack success increased from 25% to 38%.
The study found that optimized persuaders “increasingly rely on credibility-based tactics, including fabricated citations and false authoritative evidence,” according to the paper. The researchers concluded that “even when they initially reason correctly, they can be steered toward false conclusions by optimized natural language influence,” positioning persuasion robustness as “a necessary safety criterion for multi-agent and human-AI decision-making systems.”