arXiv Open Access 2023

A Call for Standardization and Validation of Text Style Transfer Evaluation

Phil Ostheimer Mayank Nagda Marius Kloft Sophie Fellenz

Lihat Sumber

Abstrak

Text Style Transfer (TST) evaluation is, in practice, inconsistent. Therefore, we conduct a meta-analysis on human and automated TST evaluation and experimentation that thoroughly examines existing literature in the field. The meta-analysis reveals a substantial standardization gap in human and automated evaluation. In addition, we also find a validation gap: only few automated metrics have been validated using human experiments. To this end, we thoroughly scrutinize both the standardization and validation gap and reveal the resulting pitfalls. This work also paves the way to close the standardization and validation gap in TST evaluation by calling out requirements to be met by future research.

Topik & Kata Kunci

cs.LG cs.CL

Penulis (4)

Phil Ostheimer

Mayank Nagda

Marius Kloft

Sophie Fellenz

Format Sitasi

APA MLA BibTeX

Ostheimer, P., Nagda, M., Kloft, M., Fellenz, S. (2023). A Call for Standardization and Validation of Text Style Transfer Evaluation. https://arxiv.org/abs/2306.00539

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2023
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓