arXiv Open Access 2020

Designing Precise and Robust Dialogue Response Evaluators

Tianyu Zhao Divesh Lala Tatsuya Kawahara

Lihat Sumber

Abstrak

Automatic dialogue response evaluator has been proposed as an alternative to automated metrics and human evaluation. However, existing automatic evaluators achieve only moderate correlation with human judgement and they are not robust. In this work, we propose to build a reference-free evaluator and exploit the power of semi-supervised training and pretrained (masked) language models. Experimental results demonstrate that the proposed evaluator achieves a strong correlation (> 0.6) with human judgement and generalizes robustly to diverse responses and corpora. We open-source the code and data in https://github.com/ZHAOTING/dialog-processing.

Topik & Kata Kunci

cs.CL

Penulis (3)

Tianyu Zhao

Divesh Lala

Tatsuya Kawahara

Format Sitasi

APA MLA BibTeX

Zhao, T., Lala, D., Kawahara, T. (2020). Designing Precise and Robust Dialogue Response Evaluators. https://arxiv.org/abs/2004.04908

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2020
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓