Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

Jonathan W. Siegel; Jinchao Xu

doi:10.1007/s00365-023-09626-4

Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

Jonathan W. Siegel^*, Jinchao Xu

^*Corresponding author for this work

Applied Mathematics and Computational Science

Research output: Contribution to journal › Article › peer-review

6 Scopus citations

Abstract

We study the variation space corresponding to a dictionary of functions in L²(Ω) for a bounded domain Ω ⊂ R^d. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.

Original language	English (US)
Pages (from-to)	1109-1132
Number of pages	24
Journal	Constructive Approximation
Volume	57
Issue number	3
DOIs	https://doi.org/10.1007/s00365-023-09626-4
State	Published - Jun 2023

Bibliographical note

Funding Information:
We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387).

Funding Information:
We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387).

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

Approximation
Function space
Neural networks

ASJC Scopus subject areas

Analysis
General Mathematics
Computational Mathematics

Access to Document

10.1007/s00365-023-09626-4

Cite this

@article{51fbab70dc8c4d8e980e45afc9d5348e,

title = "Characterization of the Variation Spaces Corresponding to Shallow Neural Networks",

abstract = "We study the variation space corresponding to a dictionary of functions in L2(Ω) for a bounded domain Ω ⊂ Rd. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.",

keywords = "Approximation, Function space, Neural networks",

author = "Siegel, {Jonathan W.} and Jinchao Xu",

note = "Funding Information: We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387). Funding Information: We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387). Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.",

year = "2023",

month = jun,

doi = "10.1007/s00365-023-09626-4",

language = "English (US)",

volume = "57",

pages = "1109--1132",

journal = "Constructive Approximation",

issn = "0176-4276",

publisher = "Springer Verlag",

number = "3",

}

TY - JOUR

T1 - Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

AU - Siegel, Jonathan W.

AU - Xu, Jinchao

N1 - Funding Information: We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387). Funding Information: We would like to thank Professors Russel Caflisch, Ronald DeVore, Weinan E, Albert Cohen, Stephan Wojtowytsch and Jason Klusowski for helpful discussions. We would also like to thank the anonymous reviewers for their helpful comments. This work was supported by the Verne M. Willaman Chair Fund at the Pennsylvania State University, and the National Science Foundation (Grant No. DMS-1819157 and DMS-2111387). Publisher Copyright: © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

PY - 2023/6

Y1 - 2023/6

N2 - We study the variation space corresponding to a dictionary of functions in L2(Ω) for a bounded domain Ω ⊂ Rd. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.

AB - We study the variation space corresponding to a dictionary of functions in L2(Ω) for a bounded domain Ω ⊂ Rd. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.

KW - Approximation

KW - Function space

KW - Neural networks

UR - http://www.scopus.com/inward/record.url?scp=85148507303&partnerID=8YFLogxK

U2 - 10.1007/s00365-023-09626-4

DO - 10.1007/s00365-023-09626-4

M3 - Article

AN - SCOPUS:85148507303

SN - 0176-4276

VL - 57

SP - 1109

EP - 1132

JO - Constructive Approximation

JF - Constructive Approximation

IS - 3

ER -

Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this