Bookmark Saya

arXiv Open Access 2023

Empirical study of the modulus as activation function in computer vision applications

Iván Vallés-Pérez Emilio Soria-Olivas Marcelino Martínez-Sober Antonio J. Serrano-López Joan Vila-Francés +1 lainnya
Lihat Sumber

Abstrak

In this work we propose a new non-monotonic activation function: the modulus. The majority of the reported research on nonlinearities is focused on monotonic functions. We empirically demonstrate how by using the modulus activation function on computer vision tasks the models generalize better than with other nonlinearities - up to a 15% accuracy increase in CIFAR100 and 4% in CIFAR10, relative to the best of the benchmark activations tested. With the proposed activation function the vanishing gradient and dying neurons problems disappear, because the derivative of the activation function is always 1 or -1. The simplicity of the proposed function and its derivative make this solution specially suitable for TinyML and hardware applications.

Topik & Kata Kunci

cs.CV cs.AI

Penulis (6)

I

Iván Vallés-Pérez

E

Emilio Soria-Olivas

M

Marcelino Martínez-Sober

A

Antonio J. Serrano-López

J

Joan Vila-Francés

J

Juan Gómez-Sanchís

Format Sitasi

APA MLA BibTeX
Vallés-Pérez, I., Soria-Olivas, E., Martínez-Sober, M., Serrano-López, A.J., Vila-Francés, J., Gómez-Sanchís, J. (2023). Empirical study of the modulus as activation function in computer vision applications. https://arxiv.org/abs/2301.05993

Akses Cepat

Lihat di Sumber
Informasi Jurnal
Tahun Terbit
2023
Bahasa
en
Sumber Database
arXiv
Akses
Open Access ✓