arXiv Open Access 2023

Power-law Dynamic arising from machine learning

Wei Chen Weitao Du Zhi-Ming Ma Qi Meng

Lihat Sumber

Abstrak

We study a kind of new SDE that was arisen from the research on optimization in machine learning, we call it power-law dynamic because its stationary distribution cannot have sub-Gaussian tail and obeys power-law. We prove that the power-law dynamic is ergodic with unique stationary distribution, provided the learning rate is small enough. We investigate its first exist time. In particular, we compare the exit times of the (continuous) power-law dynamic and its discretization. The comparison can help guide machine learning algorithm.

Topik & Kata Kunci

stat.ML cs.LG

Penulis (4)

Wei Chen

Weitao Du

Zhi-Ming Ma

Qi Meng

Format Sitasi

APA MLA BibTeX

Chen, W., Du, W., Ma, Z., Meng, Q. (2023). Power-law Dynamic arising from machine learning. https://arxiv.org/abs/2306.09624

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2023
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓