Researchers have introduced NeuroProlog, a neurosymbolic framework designed to address reliability issues in large language models’ (LLMs) mathematical reasoning capabilities, according to a paper published on arxiv.org.
According to the paper, LLMs “achieve strong performance on natural language tasks but remain unreliable in mathematical reasoning, frequently generating fluent yet logically inconsistent solutions.” NeuroProlog addresses this by compiling math word problems into executable Prolog programs with formal verification guarantees.
The framework employs what the researchers call a “Cocktail training strategy” that jointly optimizes three objectives: mathematical formula-to-rule translation, natural language-to-program synthesis, and program-answer alignment. According to the paper, “This joint supervision enables positive transfer, where symbolic grounding in formula translation directly improves compositional reasoning capabilities.”
The system includes an execution-guided decoding pipeline with “fine-grained error taxonomy that enables iterative program repair and quantifies model self-debugging capacity,” according to arxiv.org. Evaluation on the GSM8K benchmark showed that cocktail training improved accuracy over single-task baselines “with statistically significant gains for most evaluated models.”
The researchers found scale-dependent differences in error patterns: larger models exhibited “more readily correctable errors,” while smaller models showed “reduced syntactic errors but persistent semantic failures,” suggesting that model capacity influences the acquisition of reliable symbolic reasoning capabilities.