基于特征膨胀卷积模块的轻量化技术研究

doi:10.12263/DZXB.20210559

摘要/Abstract

摘要：

本文从卷积神经网络模型的网络结构入手，利用特征复用的思想，设计了高效的特征膨胀卷积模块.该模块减少了标准卷积模块的输出通道数，引入了多分支结构.通过各个分支上的廉价操作对标准卷积操作的输出特征图进行变换和融合，产生新的特征图.模块的最终输出由各个分支上生成的特征图进行合并连接得到.特征膨胀卷积模块利用特征复用思想复用模型中的特征，在降低模型计算量的同时，丰富了特征图隐含的信息，提高了模型的性能.最后，将特征膨胀卷积模块代替标准卷积模块，设计了轻量化的VGG16（Visual Geometry Group 16-Layer）模型和残差结构，并且在CIFAR数据集和ILSVRC2012（ImageNet Large Scale Visual Recognition Challenge 2012）数据集上取得了较好的分类效果.

关键词: 卷积神经网络, 轻量化, 特征复用, 特征膨胀卷积, 深度学习, 图像分类

Abstract:

This paper starts with the network structure of convolution neural network model, and uses the idea of feature reuse to design an efficient feature expansion convolution module. The module reduces the number of output channels of standard convolution module and introduces multi branch structure. Through the cheap operation on each branch, the output feature map of standard convolution operation is transformed and fused to generate a new feature map. The final output of the module is obtained by merging the feature graphs generated on each branch. The feature expansion convolution module uses the idea of feature reuse to reuse the features in the model, which not only reduces the calculation of the model, but also enriches the hidden information of the feature graph and improves the performance of the model. Finally, the feature expansion convolution module is used to replace the standard convolution module, and the lightweight VGG16 (Visual Geometry Group 16-Layer) model and residual structure are designed, and good classification results are achieved on CIFAR and ILSVRC2012 (ImageNet Large Scale Visual Recognition Challenge 2012) datasets.

Key words: convolutional neural network, lightweight, feature reuse, feature expansion convolution, deep learning, image classification

中图分类号:

TP181

许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.

Xin-zheng XU, Shan LI . Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution[J]. Acta Electronica Sinica, 2023, 51(2): 355-364.

图/表 10

图1 特征膨胀卷积模块

图2 卷积计算感受野变化图

图3 基于特征膨胀卷积模块的VGG16设计

表1 Ghost特征与Fuse特征的组合比例

分支类型	$N g$ 数	$N f$ 数	$N g$ (%)	$N f$ (%)
4	1	2	25	50
4	2	1	50	25
8	1	6	12.5	75
8	2	5	25	62.5
8	3	4	37.5	50
8	4	3	50	37.5
8	5	2	62.5	25
8	6	1	75	12.5

表1 Ghost特征与Fuse特征的组合比例

分支类型	$N g$ 数	$N f$ 数	$N g$ (%)	$N f$ (%)
4	1	2	25	50
4	2	1	50	25
8	1	6	12.5	75
8	2	5	25	62.5
8	3	4	37.5	50
8	4	3	50	37.5
8	5	2	62.5	25
8	6	1	75	12.5

表2 基于特征膨胀卷积模块的最优轻量化VGG16模型

Layer	Module	𝑚, [ $C, N g, N f$ ]
Block1	FExpand4	16×4, [1,1,2]
Block2	FExpand4	16×4, [1,1,2]
Pooling	MaxPooling
Block3	FExpand4	32×4, [1,1,2]
Block4	FExpand4	32×4, [1,1,2]
Pooling	MaxPooling
Block5	FExpand4	64×4, [1,1,2]
Block6	FExpand4	64×4, [1,1,2]
Block7	FExpand4	64×4, [1,1,2]
Pooling	MaxPooling
Block8	FExpand4	64×4, [1,1,2]
Block9	FExpand4	64×8, [1,1,6]
Block10	FExpand8	64×8, [1,1,6]
Pooling	MaxPooling
Block11	FExpand8	64×8, [1,1,6]
Block12	FExpand8	64×8, [1,1,6]
Block13	FExpand8	64×8, [1,1,6]
Pooling	AvePooling
Classfier	Fully Connected+Softmax

表2 基于特征膨胀卷积模块的最优轻量化VGG16模型

Layer	Module	𝑚, [ $C, N g, N f$ ]
Block1	FExpand4	16×4, [1,1,2]
Block2	FExpand4	16×4, [1,1,2]
Pooling	MaxPooling
Block3	FExpand4	32×4, [1,1,2]
Block4	FExpand4	32×4, [1,1,2]
Pooling	MaxPooling
Block5	FExpand4	64×4, [1,1,2]
Block6	FExpand4	64×4, [1,1,2]
Block7	FExpand4	64×4, [1,1,2]
Pooling	MaxPooling
Block8	FExpand4	64×4, [1,1,2]
Block9	FExpand4	64×8, [1,1,6]
Block10	FExpand8	64×8, [1,1,6]
Pooling	MaxPooling
Block11	FExpand8	64×8, [1,1,6]
Block12	FExpand8	64×8, [1,1,6]
Block13	FExpand8	64×8, [1,1,6]
Pooling	AvePooling
Classfier	Fully Connected+Softmax

图4 基于特征膨胀卷积模块的残差模块

表3 基于特征膨胀卷积模块的轻量化VGG16模型在CIFAR10数据集上的实验结果

	( $C, N g, N f$ )	$N f$ 比例	Weights(M)	Reduce(%)	Acc(%)
FExpand4	1,1,2	50%	3.91	74.18	92.7
FExpand4	1,2,1	25%	3.80	73.44	92.7
FExpand8	1,1,6	75%	2.02	86.28	92.5
FExpand8	1,2,5	62.5%	1.99	86.48	92.5
FExpand8	1,3,4	50%	1.96	86.68	92.4
FExpand8	1,4,3	37.5	1.94	86.82	91.9
VGG16	-	-	14.72	-	92.3

表3 基于特征膨胀卷积模块的轻量化VGG16模型在CIFAR10数据集上的实验结果

	( $C, N g, N f$ )	$N f$ 比例	Weights(M)	Reduce(%)	Acc(%)
FExpand4	1,1,2	50%	3.91	74.18	92.7
FExpand4	1,2,1	25%	3.80	73.44	92.7
FExpand8	1,1,6	75%	2.02	86.28	92.5
FExpand8	1,2,5	62.5%	1.99	86.48	92.5
FExpand8	1,3,4	50%	1.96	86.68	92.4
FExpand8	1,4,3	37.5	1.94	86.82	91.9
VGG16	-	-	14.72	-	92.3

表4 基于两种特征膨胀卷积模块的改进VGG16模型在CIFAR10数据集上的实验结果

Layer	Weights(M)	Acc(%)
j=1	2.023	92.5
j=2	2.023	92.5
j=3	2.033	92.5
j=4	2.053	92.5
j=5	2.093	92.5
j=6	2.173	92.6
j=7	2.243	92.6
j=8	2.403	92.6
j=9	2.703	92.7
j=10	3.003	92.7
j=11	3.313	92.7
j=12	3.613	92.7
j=13	3.913	92.7

表5 轻量化VGG16模型在CIFAR10和CIFAR100数据集上的测试结果

Models	Weights(M)	GMAC	CIFAR10(%)	CIFAR100(%)
VGG-16^[12]	14.72	0.316	92.3	72.3
FitNet^[36]	2.5	0.382	91.61	64.96
Highway Network^[29]	2.3	0.372	92.24	67.76
DenseNet121^[32]	15.3	0.143	94.81	80.36
ResNet-50 ^[14]	25.557	0.086	93.03	77.78
SE-Net^[34]	47.7	0.259	94.79	80.02
CondenseNet ^[35]	0.52	0.006	94.48	76.36
Ghost-VGG-16 ^[33]	7.387	0.156	93.7	68.80
Ghost-ResNet50 ^[33]	12.36	0.02	92.7	72.6
VGG16-our	2.703	0.189	92.7	73.4
ResNet50-our	5.48	0.01	93.3	78.1

表6 轻量化VGG16模型在ImageNet数据集上的测试结果 (%)

Models	Top-1 error	Top-5 error
VGG-16 ^[12]	71.93	90.67
DenseNet121^[32]	76.39	93.34
ResNet-50 ^[14]	77.15	93.29
Ghost-ResNet^[33]	75	92.3
VGG16-our	75.68	92.6
ResNet50-our	77.16	93.34

参考文献 36

1	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251.
	ZHOU F Y, JIN L P, DONG J. Review of convolutional neural network[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251. (in Chinese)
2	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
	LU H T, ZHANG Q C. Applications of deep convolutional neural network in computer vision[J]. Journal of Data Acquisition and Processing, 2016, 31(1): 1-17. (in Chinese)
3	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
4	权宇, 李志欣, 张灿龙, 等. 融合深度扩张网络和轻量化网络的目标检测模型[J]. 电子学报, 2020, 48(2): 390-397.
	QUAN Y, LI Z X, ZHANG C L, et al. Fusing deep dilated convolutions network and light-weight network for object detection[J]. Acta Electronica Sinica, 2020, 48(2): 390-397. (in Chinese)
5	赖小波, 许茂盛, 徐小媚. 多分类CNN的胶质母细胞瘤多模态MR图像分割[J]. 电子学报, 2019, 47(8): 1738-1747.
	LAI X B, XU M S, XU X M. Glioblastoma multiforme multi-modal MR images segmentation using multi-class CNN[J]. Acta Electronica Sinica, 2019, 47(8): 1738-1747. (in Chinese)
6	王琦, 谭娟. 基于人工智能技术的光学超精密检测技术[J]. 激光杂志, 2021, 42(2): 156-160.
	WANG Q, TAN J. Optical ultra-precision detection technology based on artificial intelligence technology[J]. Laser Journal, 2021, 42(2): 156-160. (in Chinese)
7	赵耀霞, 吴桐, 韩焱. 基于卷积神经网络的复杂构件内部零件装配正确性识别[J]. 电子学报, 2018, 46(8): 1983-1988.
	ZHAO Y X, WU T, HAN Y. Identifying the correctness of fit of internal components based on a convolutional neural network[J]. Acta Electronica Sinica, 2018, 46(8): 1983-1988. (in Chinese)
8	郭棉, 张锦友. 移动边缘计算环境中面向机器学习的计算迁移策略[J]. 计算机应用, 2021, 41(9): 2639-2645.
	GUO M, ZHANG J Y. Computation offloading policy for machine learning in mobile edge computing environments[J]. Journal of Computer Applications, 2021, 41(9): 2639-2645. (in Chinese)
9	VICENTE S, CARREIRA J, AGAPITO L, et al. Reconstructing pascal voc[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 41-48.
10	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]//European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
11	HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification[C]//2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1026-1034.
12	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014-09-04)[2021-05-05]. .
13	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
14	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
15	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision, 2015, 115(3): 211-252.
16	CHENG Y, WANG D, ZHOU P, et al. Model compression and acceleration for deep neural networks: The principles, progress, and challenges[J]. IEEE Signal Processing Magazine, 2018, 35(1): 126-136.
17	KWON D, KIM H, KIM J, et al. A survey of deep learning-based network anomaly detection[J]. Cluster Computing, 2019, 22(1): 949-961.
18	SINDHWANI V, SAINATH T, KUMAR S. Structured transforms for small-footprint deep learning[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2. Cambridge: MIT Press, 2015: 3088-3096.
19	CHEN W L, WILSON J T, TYREE S, et al. Compressing neural networks with the hashing trick[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37. Cambridge: JMLR, 2015: 2285-2294.
20	KARPATHY A, TODERICI G, SHETTY S, et al. Large-scale video classification with convolutional neural networks[C]//CVPR'14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1725-1732.
21	LI Y Y, BU R, SUN M C, et al. PointCNN: Convolution on X-transformed points[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2018: 828-838.
22	IOANNOU Y, ROBERTSON D, CIPOLLA R, et al. Deep roots: Improving CNN efficiency with hierarchical filter groups[EB/OL]. (2016-05-20)[2021-05-05]. .
23	朱虎明, 李佩, 焦李成, 等. 深度神经网络并行化研究综述[J]. 计算机学报, 2018, 41(8): 1861-1881.
	ZHU H M, LI P, JIAO L C, et al. Review of parallel deep neural network[J]. Chinese Journal of Computers, 2018, 41(8): 1861-1881. (in Chinese)
24	HOWARD ANDREW G, ZHU MENGLONG, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-27)[2021-05-05]. .
25	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4510-4520.
26	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 1314-1324.
27	ZAGORUYKO S, KOMODAKIS N. Wide residual networks[EB/OL]. (2016-05-23)[2021-05-05]. .
28	HUANG G, SUN Y, LIU Z, et al. Deep networks with stochastic depth[C]//European Conference on Computer Vision. Cham: Springer, 2016: 646-661.
29	SRIVASTAVA R K, GREFF K, SCHMIDHUBER J. Highway networks[EB/OL]. (2015-05-03)[2021-05-05]. .
30	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 936-944.
31	LIU Z, LI J G, SHEN Z Q, et al. Learning efficient convolutional networks through network slimming[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2755-2763.
32	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261-2269.
33	HAN K, WANG Y H, TIAN Q, et al. GhostNet: More features from cheap operations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 1577-1586.
34	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. Piscataway: IEEE, 2019: 2011-2023.
35	HUANG G, LIU S C, MAATEN L V D, et al. CondenseNet: An efficient DenseNet using learned group convolutions[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2752-2761.
36	ROMERO A, BALLAS N, KAHOU S E, et al. FitNets: Hints for thin deep nets[EB/OL]. (2014-12-19)[2021-05-05]. .

[1]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[2]	黄赟, 张帆, 郭威, 陈立, 羊光. 一种基于数据标准差的卷积神经网络量化方法[J]. 电子学报, 2023, 51(3): 639-647.
[3]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[4]	吕杭, 蒋明峰, 李杨, 张鞠成, 王志康. 基于混合时频域特征的卷积神经网络心律失常分类方法的研究[J]. 电子学报, 2023, 51(3): 701-711.
[5]	王子为, 鲁继文, 周杰. 基于自适应梯度优化的二值神经网络[J]. 电子学报, 2023, 51(2): 257-266.
[6]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[7]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[8]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.
[9]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.
[10]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[11]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[12]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[13]	丁琪, 田萱, 孙国栋. 基于注意力增强的热点感知新闻推荐模型[J]. 电子学报, 2023, 51(1): 93-104.
[14]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[15]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.