Semantic Scholar Open Access 2022 153 sitasi

When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain

Raj Sanjay Shah Kunal Chawla Dheeraj Eidnani Agam Shah Wendi Du +5 lainnya

Lihat Sumber DOI

Abstrak

Pre-trained language models have shown impressive performance on a variety of tasks and domains. Previous research on financial language models usually employs a generic training scheme to train standard model architectures, without completely leveraging the richness of the financial data. We propose a novel domain specific Financial LANGuage model (FLANG) which uses financial keywords and phrases for better masking, together with span boundary objective and in-filing objective. Additionally, the evaluation benchmarks in the field have been limited. To this end, we contribute the Financial Language Understanding Evaluation (FLUE), an open-source comprehensive suite of benchmarks for the financial domain. These include new benchmarks across 5 NLP tasks in financial domain as well as common benchmarks used in the previous research. Experiments on these benchmarks suggest that our model outperforms those in prior literature on a variety of NLP tasks. Our models, code and benchmark data will be made publicly available on Github and Huggingface.

Topik & Kata Kunci

Computer Science

Penulis (10)

Raj Sanjay Shah

Kunal Chawla

Dheeraj Eidnani

Agam Shah

Wendi Du

S. Chava

Natraj Raman

Charese Smiley

Jiaao Chen

Diyi Yang

Format Sitasi

APA MLA BibTeX

Shah, R.S., Chawla, K., Eidnani, D., Shah, A., Du, W., Chava, S. et al. (2022). When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain. https://doi.org/10.48550/arXiv.2211.00083

Akses Cepat

Lihat di Sumber doi.org/10.48550/arXiv.2211.00083

Informasi Jurnal

Tahun Terbit: 2022
Bahasa: en
Total Sitasi: 153×
Sumber Database: Semantic Scholar
DOI: 10.48550/arXiv.2211.00083
Akses: Open Access ✓