An interpretive constrained linear model for ResNet and MgNet.

Juncai He; Jinchao Xu; Lian Zhang; Jianqing Zhu

doi:10.1016/j.neunet.2023.03.011

An interpretive constrained linear model for ResNet and MgNet.

Juncai He, Jinchao Xu, Lian Zhang, Jianqing Zhu

Research output: Contribution to journal › Article › peer-review

11 Scopus citations

Abstract

We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that, compared with the original models, have fewer parameters but can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to demonstrate the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems, particularly in comparison with established networks.

Original language	English (US)
Pages (from-to)	384-392
Number of pages	9
Journal	Neural networks : the official journal of the International Neural Network Society
Volume	162
DOIs	https://doi.org/10.1016/j.neunet.2023.03.011
State	Published - Mar 20 2023

Bibliographical note

KAUST Repository Item: Exported on 2023-03-27
Acknowledgements: This work was partially supported by the Center for Computational Mathematics and Applications (CCMA) at The Pennsylvania State University, the Verne M. William Professorship Fund from The Pennsylvania State University, and the National Science Foundation, United States (Grant No. DMS-1819157 and DMS-2111387). In addition, the first and second authors are partially supported by the KAUST Baseline Research Fund, the third author is supported by Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone Project (No. HZOSWS-KCCYB-2022046), and the fourth author is supported in part by Beijing Natural Science Foundation Project (No. Z200002). The authors thank Huang Huang for his help with partial numerical experiments.

ASJC Scopus subject areas

Artificial Intelligence
Cognitive Neuroscience

Access to Document

10.1016/j.neunet.2023.03.011

Cite this

@article{31eda608ba4f47cba86956e73ca1ce60,

title = "An interpretive constrained linear model for ResNet and MgNet.",

abstract = "We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that, compared with the original models, have fewer parameters but can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to demonstrate the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems, particularly in comparison with established networks.",

author = "Juncai He and Jinchao Xu and Lian Zhang and Jianqing Zhu",

note = "KAUST Repository Item: Exported on 2023-03-27 Acknowledgements: This work was partially supported by the Center for Computational Mathematics and Applications (CCMA) at The Pennsylvania State University, the Verne M. William Professorship Fund from The Pennsylvania State University, and the National Science Foundation, United States (Grant No. DMS-1819157 and DMS-2111387). In addition, the first and second authors are partially supported by the KAUST Baseline Research Fund, the third author is supported by Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone Project (No. HZOSWS-KCCYB-2022046), and the fourth author is supported in part by Beijing Natural Science Foundation Project (No. Z200002). The authors thank Huang Huang for his help with partial numerical experiments.",

year = "2023",

month = mar,

day = "20",

doi = "10.1016/j.neunet.2023.03.011",

language = "English (US)",

volume = "162",

pages = "384--392",

journal = "Neural networks : the official journal of the International Neural Network Society",

issn = "0893-6080",

publisher = "Elsevier",

}

TY - JOUR

T1 - An interpretive constrained linear model for ResNet and MgNet.

AU - He, Juncai

AU - Xu, Jinchao

AU - Zhang, Lian

AU - Zhu, Jianqing

N1 - KAUST Repository Item: Exported on 2023-03-27 Acknowledgements: This work was partially supported by the Center for Computational Mathematics and Applications (CCMA) at The Pennsylvania State University, the Verne M. William Professorship Fund from The Pennsylvania State University, and the National Science Foundation, United States (Grant No. DMS-1819157 and DMS-2111387). In addition, the first and second authors are partially supported by the KAUST Baseline Research Fund, the third author is supported by Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone Project (No. HZOSWS-KCCYB-2022046), and the fourth author is supported in part by Beijing Natural Science Foundation Project (No. Z200002). The authors thank Huang Huang for his help with partial numerical experiments.

PY - 2023/3/20

Y1 - 2023/3/20

N2 - We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that, compared with the original models, have fewer parameters but can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to demonstrate the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems, particularly in comparison with established networks.

AB - We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that, compared with the original models, have fewer parameters but can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to demonstrate the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems, particularly in comparison with established networks.

UR - http://hdl.handle.net/10754/690584

UR - https://linkinghub.elsevier.com/retrieve/pii/S0893608023001284

U2 - 10.1016/j.neunet.2023.03.011

DO - 10.1016/j.neunet.2023.03.011

M3 - Article

C2 - 36947909

SN - 0893-6080

VL - 162

SP - 384

EP - 392

JO - Neural networks : the official journal of the International Neural Network Society

JF - Neural networks : the official journal of the International Neural Network Society

ER -

An interpretive constrained linear model for ResNet and MgNet.

Abstract

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this