arXiv Open Access 2026

Common to Whom? Regional Cultural Commonsense and LLM Bias in India

Sangmitra Madhusudan Trush Shashank More Steph Buongiorno Renata Dividino Jad Kabbara +1 lainnya
Lihat Sumber

Abstrak

Existing cultural commonsense benchmarks treat nations as monolithic, assuming uniform practices within national boundaries. But does cultural commonsense hold uniformly within a nation, or does it vary at the sub-national level? We introduce Indica, the first benchmark designed to test LLMs' ability to address this question, focusing on India - a nation of 28 states, 8 union territories, and 22 official languages. We collect human-annotated answers from five Indian regions (North, South, East, West, and Central) across 515 questions spanning 8 domains of everyday life, yielding 1,630 region-specific question-answer pairs. Strikingly, only 39.4% of questions elicit agreement across all five regions, demonstrating that cultural commonsense in India is predominantly regional, not national. We evaluate eight state-of-the-art LLMs and find two critical gaps: models achieve only 13.4%-20.9% accuracy on region-specific questions, and they exhibit geographic bias, over-selecting Central and North India as the "default" (selected 30-40% more often than expected) while under-representing East and West. Beyond India, our methodology provides a generalizable framework for evaluating cultural commonsense in any culturally heterogeneous nation, from question design grounded in anthropological taxonomy, to regional data collection, to bias measurement.

Topik & Kata Kunci

Penulis (6)

S

Sangmitra Madhusudan

T

Trush Shashank More

S

Steph Buongiorno

R

Renata Dividino

J

Jad Kabbara

A

Ali Emami

Format Sitasi

Madhusudan, S., More, T.S., Buongiorno, S., Dividino, R., Kabbara, J., Emami, A. (2026). Common to Whom? Regional Cultural Commonsense and LLM Bias in India. https://arxiv.org/abs/2601.15550

Akses Cepat

Lihat di Sumber
Informasi Jurnal
Tahun Terbit
2026
Bahasa
en
Sumber Database
arXiv
Akses
Open Access ✓