arXiv Open Access 2021

Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT

Ye Liu Wolfgang Maier Wolfgang Minker Stefan Ultes

Lihat Sumber

Abstrak

This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems. While this task was previously rendered through expensive and time-consuming human labor, we present this novel task of automatic naturalness evaluation of generated language. By fine-tuning the BERT model, our proposed naturalness evaluation method shows robust results and outperforms the baselines: support vector machines, bi-directional LSTMs, and BLEURT. In addition, the training speed and evaluation performance of naturalness model are improved by transfer learning from quality and informativeness linguistic knowledge.

Topik & Kata Kunci

cs.CL cs.AI

Penulis (4)

Ye Liu

Wolfgang Maier

Wolfgang Minker

Stefan Ultes

Format Sitasi

APA MLA BibTeX

Liu, Y., Maier, W., Minker, W., Ultes, S. (2021). Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT. https://arxiv.org/abs/2109.02938

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2021
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓