Semantic Scholar Open Access 2020 2191 sitasi

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

Xiujun Li Xi Yin Chunyuan Li Xiaowei Hu Pengchuan Zhang +7 lainnya

Abstrak

Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks. While existing methods simply concatenate image region features and text features as input to the model to be pre-trained and use self-attention to learn image-text semantic alignments in a brute force manner, in this paper, we propose a new learning method Oscar (Object-Semantics Aligned Pre-training), which uses object tags detected in images as anchor points to significantly ease the learning of alignments. Our method is motivated by the observation that the salient objects in an image can be accurately detected, and are often mentioned in the paired text. We pre-train an Oscar model on the public corpus of 6.5 million text-image pairs, and fine-tune it on downstream tasks, creating new state-of-the-arts on six well-established vision-language understanding and generation tasks.

Topik & Kata Kunci

Computer Science

Penulis (12)

Xiujun Li

Xi Yin

Chunyuan Li

Xiaowei Hu

Pengchuan Zhang

Lei Zhang

Lijuan Wang

Houdong Hu

Li Dong

Furu Wei

Yejin Choi

Jianfeng Gao

Format Sitasi

APA MLA BibTeX

Li, X., Yin, X., Li, C., Hu, X., Zhang, P., Zhang, L. et al. (2020). Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. https://doi.org/10.1007/978-3-030-58577-8_8

Akses Cepat

PDF tidak tersedia langsung

Cek di sumber asli →

Lihat di Sumber doi.org/10.1007/978-3-030-58577-8_8

Informasi Jurnal

Tahun Terbit: 2020
Bahasa: en
Total Sitasi: 2191×
Sumber Database: Semantic Scholar
DOI: 10.1007/978-3-030-58577-8_8
Akses: Open Access ✓