arXiv Open Access 2022

Formal limitations of sample-wise information-theoretic generalization bounds

Hrayr Harutyunyan Greg Ver Steeg Aram Galstyan

Lihat Sumber

Abstrak

Some of the tightest information-theoretic generalization bounds depend on the average information between the learned hypothesis and a single training example. However, these sample-wise bounds were derived only for expected generalization gap. We show that even for expected squared generalization gap no such sample-wise information-theoretic bounds exist. The same is true for PAC-Bayes and single-draw bounds. Remarkably, PAC-Bayes, single-draw and expected squared generalization gap bounds that depend on information in pairs of examples exist.

Topik & Kata Kunci

cs.LG stat.ML

Penulis (3)

Hrayr Harutyunyan

Greg Ver Steeg

Aram Galstyan

Format Sitasi

APA MLA BibTeX

Harutyunyan, H., Steeg, G.V., Galstyan, A. (2022). Formal limitations of sample-wise information-theoretic generalization bounds. https://arxiv.org/abs/2205.06915

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2022
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓