Uniform approximation rates and metric entropy of shallow neural networks

We study the approximation properties of the variation spaces corresponding to shallow neural networks with respect to the uniform norm. Specifically, we consider the spectral Barron space, which consists of the convex hull of decaying Fourier modes, and the convex hull of indicator functions of half-spaces, which corresponds to shallow neural networks with sigmoidal activation function. Up to logarithmic factors, we determine the metric entropy and nonlinear dictionary approximation rates for these spaces with respect to the uniform norm. Combined with previous results with respect to the L2-norm, this also gives the metric entropy up to logarithmic factors with respect to any Lp-norm with 1 ≤ p≤ ∞. In addition, we study the approximation rates for high-order spectral Barron spaces using shallow neural networks with ReLUk activation function. Specifically, we show that for a sufficiently high-order spectral Barron space, ReLUk networks are able to achieve an approximation rate of n-(k+1) with respect to the uniform norm.