CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations
Abstrak
In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese language examinations. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques.Furthermore, we evaluate a range of existing large language models~(LLMs), spanning from open-sourced to API-based models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities.
Topik & Kata Kunci
Penulis (11)
Jiahao Zhao
Jingwei Zhu
Minghuan Tan
Min Yang
Renhao Li
Di Yang
Chenhao Zhang
Guancheng Ye
Chengming Li
Xiping Hu
Derek F. Wong
Akses Cepat
- Tahun Terbit
- 2024
- Bahasa
- en
- Sumber Database
- arXiv
- Akses
- Open Access ✓