arXiv Open Access 2017

Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes

Takuya Akiba Shuji Suzuki Keisuke Fukuda

Lihat Sumber

Abstrak

We demonstrate that training ResNet-50 on ImageNet for 90 epochs can be achieved in 15 minutes with 1024 Tesla P100 GPUs. This was made possible by using a large minibatch size of 32k. To maintain accuracy with this large minibatch size, we employed several techniques such as RMSprop warm-up, batch normalization without moving averages, and a slow-start learning rate schedule. This paper also describes the details of the hardware and software of the system used to achieve the above performance.

Topik & Kata Kunci

cs.DC cs.CV cs.LG

Penulis (3)

Takuya Akiba

Shuji Suzuki

Keisuke Fukuda

Format Sitasi

APA MLA BibTeX

Akiba, T., Suzuki, S., Fukuda, K. (2017). Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes. https://arxiv.org/abs/1711.04325

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2017
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓