Uniform approximation rates and metric entropy of shallow neural networks

Limin Ma, Jonathan W. Siegel, Jinchao Xu

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

We study the approximation properties of the variation spaces corresponding to shallow neural networks with respect to the uniform norm. Specifically, we consider the spectral Barron space, which consists of the convex hull of decaying Fourier modes, and the convex hull of indicator functions of half-spaces, which corresponds to shallow neural networks with sigmoidal activation function. Up to logarithmic factors, we determine the metric entropy and nonlinear dictionary approximation rates for these spaces with respect to the uniform norm. Combined with previous results with respect to the L2-norm, this also gives the metric entropy up to logarithmic factors with respect to any Lp-norm with 1 ≤ p≤ ∞. In addition, we study the approximation rates for high-order spectral Barron spaces using shallow neural networks with ReLUk activation function. Specifically, we show that for a sufficiently high-order spectral Barron space, ReLUk networks are able to achieve an approximation rate of n-(k+1) with respect to the uniform norm.
Original languageEnglish (US)
JournalResearch in Mathematical Sciences
Volume9
Issue number3
DOIs
StatePublished - Sep 1 2022
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2023-02-15

Fingerprint

Dive into the research topics of 'Uniform approximation rates and metric entropy of shallow neural networks'. Together they form a unique fingerprint.

Cite this