IndicSafeEval Benchmark Exposes Safety Vulnerabilities in Multilingual AI Models

IndicSafeEval Benchmark Exposes Safety Vulnerabilities in Multilingual AI Models

Researchers have introduced IndicSafeEval, a new framework for evaluating large language model (LLM) safety in Indian languages, revealing significant vulnerabilities in current AI systems. According to arxiv.org, the benchmark combines ten safety-critical content categories with six persuasive strategies across Hindi, Bengali, Marathi, and Punjabi, generating 7,200 adversarial prompts.

The research, accepted to Findings of EMNLP 2026, found that “the model does not behave equally safely across all languages and prompt styles,” according to the paper. Instead, safety performance “depends strongly on both the languages used and the way a request is phrased using persuasive cues.” The study also observed that different risk categories exhibit varying vulnerability levels, with some harmful content types “being significantly more susceptible to persuasion-based jailbreaks than others.”

These findings highlight critical limitations in current safety evaluations, which are predominantly English-centric. According to the researchers, the work underscores “the need for multilingual and persuasion-aware benchmarking frameworks to more accurately assess real-world LLM safety.”

The research comes as related work explores alternative alignment approaches. According to arxiv.org, another study accepted at EMNLP 2026 proposes “representational similarity optimization,” which aligns LLM representations with human moral judgments and demonstrated improved adversarial robustness compared to standard behavioral alignment methods.

The IndicSafeEval implementation is available on GitHub, with the paper containing warnings about potentially offensive content.