DOAJ Open Access 2023

The Improved Stochastic Fractional Order Gradient Descent Algorithm

Yang Yang Lipo Mo Yusen Hu Fei Long

Abstrak

This paper mainly proposes some improved stochastic gradient descent (SGD) algorithms with a fractional order gradient for the online optimization problem. For three scenarios, including standard learning rate, adaptive gradient learning rate, and momentum learning rate, three new SGD algorithms are designed combining a fractional order gradient and it is shown that the corresponding regret functions are convergent at a sub-linear rate. Then we discuss the impact of the fractional order on the convergence and monotonicity and prove that the better performance can be obtained by adjusting the order of the fractional gradient. Finally, several practical examples are given to verify the superiority and validity of the proposed algorithm.

Topik & Kata Kunci

Thermodynamics Mathematics Analysis

Penulis (4)

Yang Yang

Lipo Mo

Yusen Hu

Fei Long

Format Sitasi

APA MLA BibTeX

Yang, Y., Mo, L., Hu, Y., Long, F. (2023). The Improved Stochastic Fractional Order Gradient Descent Algorithm. https://doi.org/10.3390/fractalfract7080631

Akses Cepat

PDF tidak tersedia langsung

Cek di sumber asli →

Lihat di Sumber doi.org/10.3390/fractalfract7080631

Informasi Jurnal

Tahun Terbit: 2023
Sumber Database: DOAJ
DOI: 10.3390/fractalfract7080631
Akses: Open Access ✓