arXiv Open Access 2025

Global PIQA: Evaluating Physical Commonsense Reasoning Across 100+ Languages and Cultures

Tyler A. Chang Catherine Arnett Abdelrahman Eldesokey Abdelrahman Sadallah Abeer Kashar +333 lainnya

Lihat Sumber

Abstrak

To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we present Global PIQA, a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by 335 researchers from 65 countries around the world. The 116 language varieties in Global PIQA cover five continents, 14 language families, and 23 writing systems. In the non-parallel split of Global PIQA, over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements. We find that state-of-the-art LLMs perform well on Global PIQA in aggregate, but they exhibit weaker performance in lower-resource languages (up to a 37% accuracy gap, despite random chance at 50%). Open models generally perform worse than proprietary models. Global PIQA highlights that in many languages and cultures, everyday knowledge remains an area for improvement, alongside more widely-discussed capabilities such as complex reasoning and expert knowledge. Beyond its uses for LLM evaluation, we hope that Global PIQA provides a glimpse into the wide diversity of cultures in which human language is embedded.

Topik & Kata Kunci

cs.CL

Penulis (338)

Tyler A. Chang

Catherine Arnett

Abdelrahman Eldesokey

Abdelrahman Sadallah

Abeer Kashar

Abolade Daud

Abosede Grace Olanihun

Adamu Labaran Mohammed

Adeyemi Praise

Adhikarinayum Meerajita Sharma

Aditi Gupta

Afitab Iyigun

Afonso Simplício

Ahmed Essouaied

Aicha Chorana

Akhil Eppa

Akintunde Oladipo

Akshay Ramesh

Aleksei Dorkin

Alfred Malengo Kondoro

Alham Fikri Aji

Ali Eren Çetintaş

Allan Hanbury

Alou Dembele

Alp Niksarli

Álvaro Arroyo

Amin Bajand

Amol Khanna

Ana Chkhaidze

Ana Condez

Andiswa Mkhonto

Andrew Hoblitzell

Andrew Tran

Angelos Poulis

Anirban Majumder

Anna Vacalopoulou

Annette Kuuipolani Kanahele Wong

Annika Simonsen

Anton Kovalev

Ashvanth. S

Ayodeji Joseph Lana

Barkin Kinay

Bashar Alhafni

Benedict Cibalinda Busole

Bernard Ghanem

Bharti Nathani

Biljana Stojanovska Đurić

Bola Agbonile

Bragi Bergsson

Bruce Torres Fischer

Burak Tutar

Burcu Alakuş Çınar

Cade J. Kanoniakapueo Kane

Can Udomcharoenchaikit

Catherine Arnett

Chadi Helwe

Chaithra Reddy Nerella

Chen Cecilia Liu

Chiamaka Glory Nwokolo

Cristina España-Bonet

Cynthia Amol

DaeYeop Lee

Dana Arad

Daniil Dzenhaliou

Daria Pugacheva

Dasol Choi

Daud Abolade

David Liu

David Semedo

Deborah Popoola

Deividas Mataciunas

Delphine Nyaboke

Dhyuthy Krishna Kumar

Diogo Glória-Silva

Diogo Tavares

Divyanshu Goyal

DongGeon Lee

Ebele Nwamaka Anajemba

Egonu Ngozi Grace

Elena Mickel

Elena Tutubalina

Elias Herranen

Emile Anand

Emmanuel Habumuremyi

Emuobonuvie Maria Ajiboye

Eryawan Presma Yulianrifat

Esther Adenuga

Ewa Rudnicka

Faith Olabisi Itiola

Faran Taimoor Butt

Fathima Thekkekara

Fatima Haouari

Filbert Aurelian Tjiaranata

Firas Laakom

Francesca Grasso

Francesco Orabona

Francesco Periti

Gbenga Kayode Solomon

Gia Nghia Ngo

Gloria Udhehdhe-oze

Gonçalo Martins

Gopi Naga Sai Ram Challagolla

Guijin Son

Gulnaz Abdykadyrova

Hafsteinn Einarsson

Hai Hu

Hamidreza Saffari

Hamza Zaidi

Haopeng Zhang

Harethah Abu Shairah

Harry Vuong

Hele-Andra Kuulmets

Houda Bouamor

Hwanjo Yu

Iben Nyholm Debess

İbrahim Ethem Deveci

Ikhlasul Akmal Hanif

Ikhyun Cho

Inês Calvo

Inês Vieira

Isaac Manzi

Ismail Daud

Itay Itzhak

Iuliia

Alekseenko

Ivan Belashkin

Ivan Spada

Ivan Zhelyazkov

Jacob Brinton

Jafar Isbarov

Jaka Čibej

Jan Čuhel

Jan Kocoń

Jauza Akbar Krito

Jebish Purbey

Jennifer Mickel

Jennifer Za

Jenny Kunz

Jihae Jeong

Jimena Tena Dávalos

Jinu Lee

João Magalhães

John Yi

Jongin Kim

Joseph Chataignon

Joseph Marvin Imperial

Jubeerathan Thevakumar

Judith Land

Junchen Jiang

Jungwhan Kim

Kairit Sirts

Kamesh R

Kamesh V

Kanda Patrick Tshinu

Kätriin Kukk

Kaustubh Ponkshe

Kavsar Huseynova

Ke He

Kelly Buchanan

Kengatharaiyer Sarveswaran

Kerem Zaman

Khalil Mrini

Kian Kyars

Krister Kruusmaa

Kusum Chouhan

Lainitha Krishnakumar

Laura Castro Sánchez

Laura Porrino Moscoso

Leshem Choshen

Levent Sencan

Lilja Øvrelid

Lisa Alazraki

Lovina Ehimen-Ugbede

Luheerathan Thevakumar

Luxshan Thavarasa

Mahnoor Malik

Mamadou K. Keita

Mansi Jangid

Marco De Santis

Marcos García

Marek Suppa

Mariam D'Ciofalo

Marii Ojastu

Maryam Sikander

Mausami Narayan

Maximos Skandalis

Mehak Mehak

Mehmet İlteriş Bozkurt

Melaku Bayu Workie

Menan Velayuthan

Michael Leventhal

Michał Marcińczuk

Mirna Potočnjak

Mohammadamin Shafiei

Mridul Sharma

Mrityunjaya Indoria

Muhammad Ravi Shulthan Habibi

Murat Kolić

Nada Galant

Naphat Permpredanun

Narada Maugin

Nicholas Kluge Corrêa

Nikola Ljubešić

Nirmal Thomas

Nisansa de Silva

Nisheeth Joshi

Nitish Ponkshe

Nizar Habash

Nneoma C. Udeze

Noel Thomas

Noémi Ligeti-Nagy

Nouhoum Coulibaly

Nsengiyumva Faustin

Odunayo Kareemat Buliaminu

Odunayo Ogundepo

Oghojafor Godswill Fejiro

Ogundipe Blessing Funmilola

Okechukwu God'spraise

Olanrewaju Samuel

Olaoye Deborah Oluwaseun

Olasoji Akindejoye

Olga Popova

Olga Snissarenko

Onyinye Anulika Chiemezie

Orkun Kinay

Osman Tursun

Owoeye Tobiloba Moses

Oyelade Oluwafemi Joshua

Oyesanmi Fiyinfoluwa

Pablo Gamallo

Pablo Rodríguez Fernández

Palak Arora

Pedro Valente

Peter Rupnik

Philip Oghenesuowho Ekiugbo

Pramit Sahoo

Prokopis Prokopidis

Pua Niau-Puhipau

Quadri Yahya

Rachele Mignone

Raghav Singhal

Ram Mohan Rao Kadiyala

Raphael Merx

Rapheal Afolayan

Ratnavel Rajalakshmi

Rishav Ghosh

Romina Oji

Ron Kekeha Solis

Rui Guerra

Rushikesh Zawar

Sa'ad Nasir Bashir

Saeed Alzaabi

Sahil Sandeep

Sai Pavan Batchu

SaiSandeep Kantareddy

Salsabila Zahirah Pranida

Sam Buchanan

Samuel Rutunda

Sander Land

Sarah Sulollari

Sardar Ali

Saroj Sapkota

Saulius Tautvaisas

Sayambhu Sen

Sayantani Banerjee

Sebastien Diarra

SenthilNathan. M

Sewoong Lee

Shaan Shah

Shankar Venkitachalam

Sharifa Djurabaeva

Sharon Ibejih

Shivanya Shomir Dutta

Siddhant Gupta

Silvia Paniagua Suárez

Sina Ahmadi

Sivasuthan Sukumar

Siyuan Song

Snegha A.

Sokratis Sofianopoulos

Sona Elza Simon

Sonja Benčina

Sophie Gvasalia

Sphurti Kirit More

Spyros Dragazis

Stephan P. Kaufhold

Suba. S

Sultan AlRashed

Surangika Ranathunga

Taiga Someya

Taja Kuzman Pungeršek

Tal Haklay

Tasi'u Jibril

Tatsuya Aoyama

Tea Abashidze

Terenz Jomar Dela Cruz

Terra Blevins

Themistoklis Nikas

Theresa Dora Idoko

Thu Mai Do

Tilek Chubakov

Tommaso Gargiani

Uma Rathore

Uni Johannesen

Uwuma Doris Ugwu

Vallerie Alexandra Putra

Vanya Bannihatti Kumar

Varsha Jeyarajalingam

Varvara Arzt

Vasudevan Nedumpozhimana

Viktoria Ondrejova

Viktoryia Horbik

Vishnu Vardhan Reddy Kummitha

Vuk Dinić

Walelign Tewabe Sewunetie

Winston Wu

Xiaojing Zhao

Yacouba Diarra

Yaniv Nikankin

Yash Mathur

Yixi Chen

Yiyuan Li

Yolanda Xavier

Yonatan Belinkov

Yusuf Ismail Abayomi

Zaid Alyafeai

Zhengyang Shan

Zhi Rui Tam

Zilu Tang

Zuzana Nadova

Baber Abbasi

Stella Biderman

David Stap

Duygu Ataman

Fabian Schmidt

Hila Gonen

Jiayi Wang

David Ifeoluwa Adelani

Format Sitasi

APA MLA BibTeX

Chang, T.A., Arnett, C., Eldesokey, A., Sadallah, A., Kashar, A., Daud, A. et al. (2025). Global PIQA: Evaluating Physical Commonsense Reasoning Across 100+ Languages and Cultures. https://arxiv.org/abs/2510.24081

@article{chang2025, author = {Tyler A. Chang and Catherine Arnett and Abdelrahman Eldesokey and Abdelrahman Sadallah and Abeer Kashar and Abolade Daud and Abosede Grace Olanihun and Adamu Labaran Mohammed and Adeyemi Praise and Adhikarinayum Meerajita Sharma and Aditi Gupta and Afitab Iyigun and Afonso Simplício and Ahmed Essouaied and Aicha Chorana and Akhil Eppa and Akintunde Oladipo and Akshay Ramesh and Aleksei Dorkin and Alfred Malengo Kondoro and Alham Fikri Aji and Ali Eren Çetintaş and Allan Hanbury and Alou Dembele and Alp Niksarli and Álvaro Arroyo and Amin Bajand and Amol Khanna and Ana Chkhaidze and Ana Condez and Andiswa Mkhonto and Andrew Hoblitzell and Andrew Tran and Angelos Poulis and Anirban Majumder and Anna Vacalopoulou and Annette Kuuipolani Kanahele Wong and Annika Simonsen and Anton Kovalev and Ashvanth. S and Ayodeji Joseph Lana and Barkin Kinay and Bashar Alhafni and Benedict Cibalinda Busole and Bernard Ghanem and Bharti Nathani and Biljana Stojanovska Đurić and Bola Agbonile and Bragi Bergsson and Bruce Torres Fischer and Burak Tutar and Burcu Alakuş Çınar and Cade J. Kanoniakapueo Kane and Can Udomcharoenchaikit and Catherine Arnett and Chadi Helwe and Chaithra Reddy Nerella and Chen Cecilia Liu and Chiamaka Glory Nwokolo and Cristina España-Bonet and Cynthia Amol and DaeYeop Lee and Dana Arad and Daniil Dzenhaliou and Daria Pugacheva and Dasol Choi and Daud Abolade and David Liu and David Semedo and Deborah Popoola and Deividas Mataciunas and Delphine Nyaboke and Dhyuthy Krishna Kumar and Diogo Glória-Silva and Diogo Tavares and Divyanshu Goyal and DongGeon Lee and Ebele Nwamaka Anajemba and Egonu Ngozi Grace and Elena Mickel and Elena Tutubalina and Elias Herranen and Emile Anand and Emmanuel Habumuremyi and Emuobonuvie Maria Ajiboye and Eryawan Presma Yulianrifat and Esther Adenuga and Ewa Rudnicka and Faith Olabisi Itiola and Faran Taimoor Butt and Fathima Thekkekara and Fatima Haouari and Filbert Aurelian Tjiaranata and Firas Laakom and Francesca Grasso and Francesco Orabona and Francesco Periti and Gbenga Kayode Solomon and Gia Nghia Ngo and Gloria Udhehdhe-oze and Gonçalo Martins and Gopi Naga Sai Ram Challagolla and Guijin Son and Gulnaz Abdykadyrova and Hafsteinn Einarsson and Hai Hu and Hamidreza Saffari and Hamza Zaidi and Haopeng Zhang and Harethah Abu Shairah and Harry Vuong and Hele-Andra Kuulmets and Houda Bouamor and Hwanjo Yu and Iben Nyholm Debess and İbrahim Ethem Deveci and Ikhlasul Akmal Hanif and Ikhyun Cho and Inês Calvo and Inês Vieira and Isaac Manzi and Ismail Daud and Itay Itzhak and Iuliia and Alekseenko and Ivan Belashkin and Ivan Spada and Ivan Zhelyazkov and Jacob Brinton and Jafar Isbarov and Jaka Čibej and Jan Čuhel and Jan Kocoń and Jauza Akbar Krito and Jebish Purbey and Jennifer Mickel and Jennifer Za and Jenny Kunz and Jihae Jeong and Jimena Tena Dávalos and Jinu Lee and João Magalhães and John Yi and Jongin Kim and Joseph Chataignon and Joseph Marvin Imperial and Jubeerathan Thevakumar and Judith Land and Junchen Jiang and Jungwhan Kim and Kairit Sirts and Kamesh R and Kamesh V and Kanda Patrick Tshinu and Kätriin Kukk and Kaustubh Ponkshe and Kavsar Huseynova and Ke He and Kelly Buchanan and Kengatharaiyer Sarveswaran and Kerem Zaman and Khalil Mrini and Kian Kyars and Krister Kruusmaa and Kusum Chouhan and Lainitha Krishnakumar and Laura Castro Sánchez and Laura Porrino Moscoso and Leshem Choshen and Levent Sencan and Lilja Øvrelid and Lisa Alazraki and Lovina Ehimen-Ugbede and Luheerathan Thevakumar and Luxshan Thavarasa and Mahnoor Malik and Mamadou K. Keita and Mansi Jangid and Marco De Santis and Marcos García and Marek Suppa and Mariam D'Ciofalo and Marii Ojastu and Maryam Sikander and Mausami Narayan and Maximos Skandalis and Mehak Mehak and Mehmet İlteriş Bozkurt and Melaku Bayu Workie and Menan Velayuthan and Michael Leventhal and Michał Marcińczuk and Mirna Potočnjak and Mohammadamin Shafiei and Mridul Sharma and Mrityunjaya Indoria and Muhammad Ravi Shulthan Habibi and Murat Kolić and Nada Galant and Naphat Permpredanun and Narada Maugin and Nicholas Kluge Corrêa and Nikola Ljubešić and Nirmal Thomas and Nisansa de Silva and Nisheeth Joshi and Nitish Ponkshe and Nizar Habash and Nneoma C. Udeze and Noel Thomas and Noémi Ligeti-Nagy and Nouhoum Coulibaly and Nsengiyumva Faustin and Odunayo Kareemat Buliaminu and Odunayo Ogundepo and Oghojafor Godswill Fejiro and Ogundipe Blessing Funmilola and Okechukwu God'spraise and Olanrewaju Samuel and Olaoye Deborah Oluwaseun and Olasoji Akindejoye and Olga Popova and Olga Snissarenko and Onyinye Anulika Chiemezie and Orkun Kinay and Osman Tursun and Owoeye Tobiloba Moses and Oyelade Oluwafemi Joshua and Oyesanmi Fiyinfoluwa and Pablo Gamallo and Pablo Rodríguez Fernández and Palak Arora and Pedro Valente and Peter Rupnik and Philip Oghenesuowho Ekiugbo and Pramit Sahoo and Prokopis Prokopidis and Pua Niau-Puhipau and Quadri Yahya and Rachele Mignone and Raghav Singhal and Ram Mohan Rao Kadiyala and Raphael Merx and Rapheal Afolayan and Ratnavel Rajalakshmi and Rishav Ghosh and Romina Oji and Ron Kekeha Solis and Rui Guerra and Rushikesh Zawar and Sa'ad Nasir Bashir and Saeed Alzaabi and Sahil Sandeep and Sai Pavan Batchu and SaiSandeep Kantareddy and Salsabila Zahirah Pranida and Sam Buchanan and Samuel Rutunda and Sander Land and Sarah Sulollari and Sardar Ali and Saroj Sapkota and Saulius Tautvaisas and Sayambhu Sen and Sayantani Banerjee and Sebastien Diarra and SenthilNathan. M and Sewoong Lee and Shaan Shah and Shankar Venkitachalam and Sharifa Djurabaeva and Sharon Ibejih and Shivanya Shomir Dutta and Siddhant Gupta and Silvia Paniagua Suárez and Sina Ahmadi and Sivasuthan Sukumar and Siyuan Song and Snegha A. and Sokratis Sofianopoulos and Sona Elza Simon and Sonja Benčina and Sophie Gvasalia and Sphurti Kirit More and Spyros Dragazis and Stephan P. Kaufhold and Suba. S and Sultan AlRashed and Surangika Ranathunga and Taiga Someya and Taja Kuzman Pungeršek and Tal Haklay and Tasi'u Jibril and Tatsuya Aoyama and Tea Abashidze and Terenz Jomar Dela Cruz and Terra Blevins and Themistoklis Nikas and Theresa Dora Idoko and Thu Mai Do and Tilek Chubakov and Tommaso Gargiani and Uma Rathore and Uni Johannesen and Uwuma Doris Ugwu and Vallerie Alexandra Putra and Vanya Bannihatti Kumar and Varsha Jeyarajalingam and Varvara Arzt and Vasudevan Nedumpozhimana and Viktoria Ondrejova and Viktoryia Horbik and Vishnu Vardhan Reddy Kummitha and Vuk Dinić and Walelign Tewabe Sewunetie and Winston Wu and Xiaojing Zhao and Yacouba Diarra and Yaniv Nikankin and Yash Mathur and Yixi Chen and Yiyuan Li and Yolanda Xavier and Yonatan Belinkov and Yusuf Ismail Abayomi and Zaid Alyafeai and Zhengyang Shan and Zhi Rui Tam and Zilu Tang and Zuzana Nadova and Baber Abbasi and Stella Biderman and David Stap and Duygu Ataman and Fabian Schmidt and Hila Gonen and Jiayi Wang and David Ifeoluwa Adelani}, title = {Global PIQA: Evaluating Physical Commonsense Reasoning Across 100+ Languages and Cultures}, year = {2025}, journal = {arXiv}, url = {https://arxiv.org/abs/2510.24081}, }

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2025
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓