Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution

XU Xin-zheng; LI Shan

doi:10.12263/DZXB.20210559

您当前的位置：

首页 >

文章列表页 >

Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution

PAPERS | 更新时间：2025-12-08

- Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution
- ACTA ELECTRONICA SINICA Vol. 51, Issue 2, Pages: 355-364(2023)
- 作者机构：
  
  1.中国矿业大学计算机科学与技术学院，江苏徐州 221116
  2.教育部矿山数字化工程中心，江苏徐州 221116
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20210559
  CLC： TP181
- Received：05 May 2021，
  
  Revised：2021-07-21，
  
  Published：25 February 2023
- 稿件说明：
移动端阅览
许新征,李杉.基于特征膨胀卷积模块的轻量化技术研究[J].电子学报,2023,51(02):355-364.

XU Xin-zheng,LI Shan.Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution[J].ACTA ELECTRONICA SINICA,2023,51(02):355-364.
许新征,李杉.基于特征膨胀卷积模块的轻量化技术研究[J].电子学报,2023,51(02):355-364. DOI： 10.12263/DZXB.20210559.

XU Xin-zheng,LI Shan.Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution[J].ACTA ELECTRONICA SINICA,2023,51(02):355-364. DOI： 10.12263/DZXB.20210559.

摘要

本文从卷积神经网络模型的网络结构入手，利用特征复用的思想，设计了高效的特征膨胀卷积模块.该模块减少了标准卷积模块的输出通道数，引入了多分支结构.通过各个分支上的廉价操作对标准卷积操作的输出特征图进行变换和融合，产生新的特征图.模块的最终输出由各个分支上生成的特征图进行合并连接得到.特征膨胀卷积模块利用特征复用思想复用模型中的特征，在降低模型计算量的同时，丰富了特征图隐含的信息，提高了模型的性能.最后，将特征膨胀卷积模块代替标准卷积模块，设计了轻量化的VGG16（Visual Geometry Group 16-Layer）模型和残差结构，并且在CIFAR数据集和ILSVRC2012（ImageNet Large Scale Visual Recognition Challenge 2012）数据集上取得了较好的分类效果.

Abstract

This paper starts with the network structure of convolution neural network model

and uses the idea of feature reuse to design an efficient feature expansion convolution module. The module reduces the number of output channels of standard convolution module and introduces multi branch structure. Through the cheap operation on each branch

the output feature map of standard convolution operation is transformed and fused to generate a new feature map. The final output of the module is obtained by merging the feature graphs generated on each branch. The feature expansion convolution module uses the idea of feature reuse to reuse the features in the model

which not only reduces the calculation of the model

but also enriches the hidden information of the feature graph and improves the performance of the model. Finally

the feature expansion convolution module is used to replace the standard convolution module

and the lightweight VGG16 (Visual Geometry Group 16-Layer) model and residual structure are designed

and good classification results are achieved on CIFAR and ILSVRC2012 (ImageNet Large Scale Visual Recognition Challenge 2012) datasets.

关键词

Keywords

references

周飞燕 , 金林鹏 , 董军 . 卷积神经网络研究综述 [J]. 计算机学报 , 2017 , 40 ( 6 ): 1229 - 1251 .

ZHOU F Y , JIN L P , DONG J . Review of convolutional neural network [J]. Chinese Journal of Computers , 2017 , 40 ( 6 ): 1229 - 1251 . (in Chinese)

卢宏涛 , 张秦川 . 深度卷积神经网络在计算机视觉中的应用研究综述 [J]. 数据采集与处理 , 2016 , 31 ( 1 ): 1 - 17 .

LU H T , ZHANG Q C . Applications of deep convolutional neural network in computer vision [J]. Journal of Data Acquisition and Processing , 2016 , 31 ( 1 ): 1 - 17 . (in Chinese)

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

权宇 , 李志欣 , 张灿龙 , 等 . 融合深度扩张网络和轻量化网络的目标检测模型 [J]. 电子学报 , 2020 , 48 ( 2 ): 390 - 397 .

QUAN Y , LI Z X , ZHANG C L , et al . Fusing deep dilated convolutions network and light-weight network for object detection [J]. Acta Electronica Sinica , 2020 , 48 ( 2 ): 390 - 397 . (in Chinese)

赖小波 , 许茂盛 , 徐小媚 . 多分类CNN的胶质母细胞瘤多模态MR图像分割 [J]. 电子学报 , 2019 , 47 ( 8 ): 1738 - 1747 .

LAI X B , XU M S , XU X M . Glioblastoma multiforme multi-modal MR images segmentation using multi-class CNN [J]. Acta Electronica Sinica , 2019 , 47 ( 8 ): 1738 - 1747 . (in Chinese)

王琦 , 谭娟 . 基于人工智能技术的光学超精密检测技术 [J]. 激光杂志 , 2021 , 42 ( 2 ): 156 - 160 .

WANG Q , TAN J . Optical ultra-precision detection technology based on artificial intelligence technology [J]. Laser Journal , 2021 , 42 ( 2 ): 156 - 160 . (in Chinese)

赵耀霞 , 吴桐 , 韩焱 . 基于卷积神经网络的复杂构件内部零件装配正确性识别 [J]. 电子学报 , 2018 , 46 ( 8 ): 1983 - 1988 .

ZHAO Y X , WU T , HAN Y . Identifying the correctness of fit of internal components based on a convolutional neural network [J]. Acta Electronica Sinica , 2018 , 46 ( 8 ): 1983 - 1988 . (in Chinese)

郭棉 , 张锦友 . 移动边缘计算环境中面向机器学习的计算迁移策略 [J]. 计算机应用 , 2021 , 41 ( 9 ): 2639 - 2645 .

GUO M , ZHANG J Y . Computation offloading policy for machine learning in mobile edge computing environments [J]. Journal of Computer Applications , 2021 , 41 ( 9 ): 2639 - 2645 . (in Chinese)

VICENTE S , CARREIRA J , AGAPITO L , et al . Reconstructing pascal voc [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2014 : 41 - 48 .

LIN T Y , MAIRE M , BELONGIE S , et al . Microsoft COCO: Common objects in context [C]// European Conference on Computer Vision . Cham : Springer , 2014 : 740 - 755 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification [C]// 2015 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2015 : 1026 - 1034 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL]. ( 2014-09-04 )[ 2021-05-05 ]. https://arxiv.org/abs/1409.1556 https://arxiv.org/abs/1409.1556 .

SZEGEDY C , LIU W , JIA Y Q , et al . Going deeper with convolutions [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2015 : 1 - 9 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 770 - 778 .

RUSSAKOVSKY O , DENG J , SU H , et al . ImageNet large scale visual recognition challenge [J]. International Journal of Computer Vision , 2015 , 115 ( 3 ): 211 - 252 .

CHENG Y , WANG D , ZHOU P , et al . Model compression and acceleration for deep neural networks: The principles, progress, and challenges [J]. IEEE Signal Processing Magazine , 2018 , 35 ( 1 ): 126 - 136 .

KWON D , KIM H , KIM J , et al . A survey of deep learning-based network anomaly detection [J]. Cluster Computing , 2019 , 22 ( 1 ): 949 - 961 .

SINDHWANI V , SAINATH T , KUMAR S . Structured transforms for small-footprint deep learning [C]// Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 . Cambridge : MIT Press , 2015 : 3088 - 3096 .

CHEN W L , WILSON J T , TYREE S , et al . Compressing neural networks with the hashing trick [C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 . Cambridge : JMLR , 2015 : 2285 - 2294 .

KARPATHY A , TODERICI G , SHETTY S , et al . Large-scale video classification with convolutional neural networks [C]// CVPR'14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus : IEEE , 2014 : 1725 - 1732 .

LI Y Y , BU R , SUN M C , et al . PointCNN: Convolution on X-transformed points [C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems . Red Hook : Curran Associates Inc , 2018 : 828 - 838 .

IOANNOU Y , ROBERTSON D , CIPOLLA R , et al . Deep roots: Improving CNN efficiency with hierarchical filter groups [EB/OL]. ( 2016-05-20 )[ 2021-05-05 ]. https://arxiv.org/abs/1605.06489 https://arxiv.org/abs/1605.06489 .

朱虎明 , 李佩 , 焦李成 , 等 . 深度神经网络并行化研究综述 [J]. 计算机学报 , 2018 , 41 ( 8 ): 1861 - 1881 .

ZHU H M , LI P , JIAO L C , et al . Review of parallel deep neural network [J]. Chinese Journal of Computers , 2018 , 41 ( 8 ): 1861 - 1881 . (in Chinese)

HOWARD ANDREW G , ZHU MENGLONG , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. ( 2017-04-27 )[ 2021-05-05 ]. http://arxiv.org/abs/1704.04861 http://arxiv.org/abs/1704.04861 .

SANDLER M , HOWARD A , ZHU M L , et al . MobileNetV2: inverted residuals and linear bottlenecks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 4510 - 4520 .

HOWARD A , SANDLER M , CHEN B , et al . Searching for MobileNetV3 [C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 1314 - 1324 .

ZAGORUYKO S , KOMODAKIS N . Wide residual networks [EB/OL]. ( 2016-05-23 )[ 2021-05-05 ]. https://arxiv.org/abs/1605. 07146 https://arxiv.org/abs/1605.07146 .

HUANG G , SUN Y , LIU Z , et al . Deep networks with stochastic depth [C]// European Conference on Computer Vision . Cham : Springer , 2016 : 646 - 661 .

SRIVASTAVA R K , GREFF K , SCHMIDHUBER J . Highway networks [EB/OL]. ( 2015-05-03 )[ 2021-05-05 ]. https://arxiv.org/abs/1505.00387 https://arxiv.org/abs/1505.00387 .

LIN T Y , DOLLÁR P , GIRSHICK R , et al . Feature pyramid networks for object detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 936 - 944 .

LIU Z , LI J G , SHEN Z Q , et al . Learning efficient convolutional networks through network slimming [C]// 2017 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2017 : 2755 - 2763 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 2261 - 2269 .

HAN K , WANG Y H , TIAN Q , et al . GhostNet: More features from cheap operations [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 1577 - 1586 .

HU J , SHEN L , ALBANIE S , et al . Squeeze-and-excitation networks [C]// IEEE Transactions on Pattern Analysis and Machine Intelligence . Piscataway : IEEE , 2019 : 2011 - 2023 .

HUANG G , LIU S C , MAATEN L V D , et al . CondenseNet: An efficient DenseNet using learned group convolutions [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 2752 - 2761 .

ROMERO A , BALLAS N , KAHOU S E , et al . FitNets: Hints for thin deep nets [EB/OL]. ( 2014-12-19 )[ 2021-05-05 ]. https://arxiv.org/abs/1412.6550 https://arxiv.org/abs/1412.6550 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

Image Classification Algorithm Based on Coordinate Importance Pooling and Decoupled Class Alignment Distillation

MalMKNet: A Multi-Scale Convolutional Neural Network Used for Malware Classification

Research on Lightweight Image Classification Algorithm Based on Multi-Branch Bottleneck Structure

Related Author

Xin-zheng XU

Shan LI

WANG Ying

GAO Lan

ZHANG Zhe

LIU Xin

WU Yi-xiong

ZHANG Wei-gong

Related Institution

School of Computer Science and Technology， China University of Mining and Technology

College of Information Engineering, Capital Normal University

School of Mathematical Science, Capital Normal University

Faculty of Software Technologics, Shanxi Agricultural University

Center for Image and Information Processing, Xi’an University of Posts and Telecommunications

⁰