英文论文
-
文献类型
-
Article
-
题名
-
Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks With Soft-Thresholding
-
作者
-
Xiong, Chunyan; Zhang, Chaoxing; Lu, Mengli; Yu, Xiaotong; Cao, Jian; Chen, Zhong; Guo, Di; Qu, Xiaobo
-
作者单位
-
[Xiong, Chunyan; Zhang, Chaoxing; Lu, Mengli; Yu, Xiaotong; Cao, Jian; Chen, Zhong; Qu, Xiaobo] Xiamen Univ, Fujian Prov Key Lab Plasma & Magnet Reson, Sch Elect Sci & Engn, Inst Elect & Acoust, Xiamen 361104, Peoples R China. [Guo, Di] Univ Technol, Sch Comp & Informat Engn, Xiamen 361024, Peoples R China.
-
通讯作者地址
-
Xiamen Univ, Fujian Prov Key Lab Plasma & Magnet Reson, Sch Elect Sci & Engn, Inst Elect & Acoust, Xiamen 361104, Peoples R China.
-
Email
-
quxiaobo@xmu.edu.cn
-
ResearchID
-
-
ORCID
-
-
期刊名称
-
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
-
出版社
-
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
-
ISSN
-
2162-237X
-
出版信息
-
2025-02, 36 (2):3423-3435.
-
JCR
-
-
影响因子
-
-
ISBN
-
-
基金
-
National Natural Science Foundation [61971361, 62122064, 62331021, 62371410]; Natural Science Foundation of Fujian Province of China [2023J02005, 2021J011184]; President Fund of Xiamen University [20720220063]; Xiamen University Nanqiang Outstanding Talents Program
-
会议名称
-
-
会议地点
-
-
会议开始日期
-
-
会议结束日期
-
-
关键词
-
Artificial neural networks; Convolution; Training; Kernel; Noise reduction; Noise measurement; Learning systems; Convex optimization; nonconvexity; soft-thresholding; strong duality
-
摘要
-
Soft-thresholding has been widely used in neural networks. Its basic network structure is a two-layer convolution neural network with soft-thresholding. Due to the network's nature of nonlinear and nonconvex, the training process heavily depends on an appropriate initialization of network parameters, resulting in the difficulty of obtaining a globally optimal solution. To address this issue, a convex dual network is designed here. We theoretically analyze the network convexity and prove that the strong duality holds. Extensive results on both simulation and real-world datasets show that strong duality holds, the dual network does not depend on initialization and optimizer, and enables faster convergence than the state-of-the-art two-layer network. This work provides a new way to convexify soft-thresholding neural networks. Furthermore, the convex dual network model of a deep soft-thresholding network with a parallel structure is deduced.
-
一级学科
-
Computer Science, Artificial Intelligence; Computer Science, Hardware & Architecture; Computer Science, Theory & Methods; Engineering, Electrical & Electronic
-
WOS入藏号
-
WOS:001416736600035
-
EI收录号
-
20240615527057
-
DOI
-
10.1109/TNNLS.2024.3353795
-
ESI
-
-
收录于
-
SCIE, EI
-
返回