Researchers have introduced multiple new benchmarks aimed at rigorously evaluating large language model capabilities across specialized domains, according to four papers published on arxiv.org on August 12, 2026.
QuArch evaluates LLM reasoning in computer architecture, according to a paper by Shvetank Prakash and colleagues. The benchmark tests models’ understanding of technical concepts in this specialized field.
CORDA (Conditioned Ordering and Ranked Directive Adherence) addresses hierarchical harm-centered moral reasoning, according to arxiv.org. The benchmark tests 90 moral dilemmas across four ethical frameworks including trolley-style cases, medical trade-offs, and resource allocation scenarios. Testing ten instruction-tuned models from seven providers, researchers found that 9 of 10 models prioritized “avoidance of direct personal harm over reducing overall harm,” demonstrating a “strong deontological default.” According to the paper, models “perform more reliably on categorical harm-avoidance rules, such as avoiding killing, than on outcome-based comparisons.”
MonitorBench evaluates chain-of-thought monitorability in LLMs, according to Han Wang and colleagues. The benchmark provides 1,514 test instances across 19 tasks spanning 7 categories, designed to assess when chains of thought faithfully reflect the factors driving model behavior.
SciVisAgentBench focuses on scientific data analysis and visualization agents, according to Kuangshi Ai and colleagues. The benchmark aims to provide “a principled and reproducible” evaluation framework for systems that translate natural-language intent into scientific visualization tasks.