Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计

袁海英; 成君鹏; 曾智勇; 武延瑞

doi:10.12263/DZXB.20211671

您当前的位置：

首页 >

文章列表页 >

Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计

学术论文 | 更新时间：2025-07-02

- Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计
- Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net
- 电子学报 2023年51卷第1期页码：180-191
- 作者机构：
  
  北京工业大学信息学部，北京 100124
- 作者简介：
  
  [ "袁海英女，1976年出生于四川阆中.北京工业大学信息学部副教授.主要研究方向为面向人工智能应用的高能效计算芯片系统、面向信号检测与信息处理的嵌入式系统、电子系统容错与通信总线技术.E-mail: yhycn@126.com" ]
  [ "成君鹏男，1995年出生于江苏盐城.北京工业大学信息学部硕士研究生.主要研究方向为轻量级卷积神经网络建模技术.E-mail: chengjp@emails.bjut.edu.cn" ]
  [ "曾智勇男，1997年出生于北京.北京工业大学信息学部硕士研究生.主要研究方向为基于FPGA的卷积神经网络加速器架构设计.E-mail: m_x_zy@126.com" ]
  [ "武延瑞男，1996年出生于河北衡水.北京工业大学信息学部硕士研究生.主要研究方向为医学影像智能处理技术.E-mail: 15226525376@163.com" ]
- 基金信息：
  
  国家自然科学基金(61001049);北京市自然科学基金(4172010)
- DOI：10.12263/DZXB.20211671
  中图分类号： TP391;
- 收稿：2021-12-18，
  
  修回：2022-03-07，
  
  纸质出版：2023-01-25
- 稿件说明：
移动端阅览
袁海英,成君鹏,曾智勇等.Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J].电子学报,2023,51(01):180-191.

YUAN Hai-ying,CHENG Jun-peng,ZENG Zhi-yong,et al.Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net[J].ACTA ELECTRONICA SINICA,2023,51(01):180-191.
袁海英,成君鹏,曾智勇等.Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J].电子学报,2023,51(01):180-191. DOI： 10.12263/DZXB.20211671.

YUAN Hai-ying,CHENG Jun-peng,ZENG Zhi-yong,et al.Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net[J].ACTA ELECTRONICA SINICA,2023,51(01):180-191. DOI： 10.12263/DZXB.20211671.

摘要

针对深度卷积神经网络难以部署到资源受限的端侧设备这一问题，本文提出一种高效精简的轻量化卷积神经网络Mobile_BLNet，在模型规模、计算量和性能之间取得了良好的平衡.该网络引入深度可分离卷积和倒残差结构，通过合理分配不同分支的运算量缩减模型规模并节省大量计算资源；采用通道剪枝操作压缩网络模型，基于占总和比值方法裁剪对模型贡献度低的卷积通道，在相同压缩效果情况下提升了分类准确率；基于通道裁剪情况重构网络，进一步降低模型所需计算资源.实验结果表明，Mobile_BLNet结构精简、性能优异，在CIFAR-10/CIFAR-100数据集上以0.1 M/0.3 M参数量、9.6 M/12.7 M浮点计算量获得91.2%/71.5%分类准确率；在Food101/ImageNet数据集上以1.0 M/2.1 M参数量、203.0 M/249.6 M浮点计算量获得82.8%/70.9%分类准确率，满足轻量化卷积神经网络的端侧硬件高能效部署需求.

Abstract

Since it is difficult for deep convolutional neural network to be deployed to terminal equipment with limited resources

this paper proposes an efficient

compact

and lightweight network Mobile_BLNet

which achieves a good balance between model size

computation

and performance. The network uses depthwise separable convolution and inverse residual structure

reduces the scale of the model and saves a lot of computing resources by reasonably allocating the amount of computation of different branches. The total ratio method is used to prune the convolution channel with low contribution

which has excellent performance under the same compression effect. Model reconstruction is based on the clipping

which further reduces the computational resources. The experimental results show that Mobile_BLNet has excellent performance. On CIFAR-10/CIFAR-100 dataset

91.2%/71.5% accuracy is obtained with 0.1 M/0.3 M parameters and 9.6 M/12.7 M floating point operations. On Food101/ImageNet dataset

82.8%/70.9% accuracy is obtained with 1.0 M/2.1 M parameters and 203.0 M/249.6 M floating point operations. The network meets the requirements of energy-efficient and lightweight hardware deployment.

关键词

Keywords

references

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [C]// Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe Nevada : ACM , 2012 : 1097 - 1105 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL]. ( 2014-09-04 )[ 2021-12 ]. https://arxiv.org/abs/1409.1556 https://arxiv.org/abs/1409.1556 .

SZEGEDY C , LIU W , JIA Y Q , et al . Going deeper with convolutions [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston : IEEE , 2015 : 1 - 9 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 770 - 778 .

HUANG H H , ZHOU P , LI Y , et al . A lightweight attention-based CNN model for efficient gait recognition with wearable IMU sensors [J]. Sensors(Basel) , 2021 , 21 ( 8 ): 2866 .

SHUVO S B , ALI S N , SWAPNIL S I , et al . A lightweight CNN model for detecting respiratory diseases from lung auscultation sounds using EMD-CWT-based hybrid scalogram [J]. IEEE Journal of Biomedical and Health Informatics , 2021 , 25 ( 7 ): 2595 - 2603 .

IANDOLA F N , HAN S , MOSKEWICZ M W , et al . SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0 . 5 MB model size[EB/OL]. ( 2016-02-24 )[ 2021-12 ]. https://arxiv.org/abs/1602.07360 https://arxiv.org/abs/1602.07360 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. ( 2017-04-17 )[ 2021-12 ]. https://arxiv.org/abs/1704.04861 https://arxiv.org/abs/1704.04861 .

SANDLER M , HOWARD A , ZHU M L , et al . MobileNetV2: Inverted residuals and linear bottlenecks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 4510 - 4520 .

ZHANG X Y , ZHOU X Y , LIN M X , et al . ShuffleNet: An extremely efficient convolutional neural network for mobile devices [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 6848 - 6856 .

MA N N , ZHANG X Y , ZHENG H T , et al . ShuffleNet V2: Practical guidelines for efficient CNN architecture design [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 122 - 138 .

TAN M X , LE Q V . EfficientNet: Rethinking model scaling for convolutional neural networks [C]// Proceedings of the 36th International Conference on Machine Learning . Long Beach : MLResearch Press , 2019 , 97 : 6105 - 6114 .

CHEN C F , FAN Q F , MALLINAR N , et al . Big-Little Net: An efficient multi-scale feature representation for visual and speech recognition [EB/OL]. ( 2018-07-10 )[ 2021-12 ]. https://arxiv.org/abs/1807.03848 https://arxiv.org/abs/1807.03848 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 2261 - 2269 .

DENTON E , ZAREMBA W , BRUNA J , et al . Exploiting linear structure within convolutional networks for efficient evaluation [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems . New York : ACM , 2014 : 1269 - 1277 .

DONG X Y , HUANG J S , YANG Y , et al . More is less: A more complicated network with less inference complexity [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 1895 - 1903 .

LI H , KADAV A , DURDANOVIC I , et al . Pruning filters for efficient ConvNets [EB/OL]. ( 2016-08-31 )[ 2021-12 ]. https://arxiv.org/abs/1608.08710 https://arxiv.org/abs/1608.08710 .

TANG Y H , WANG Y H , XU Y X , et al . Manifold regularized dynamic network pruning [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 5016 - 5026 .

HAN S , POOL J , TRAN J , et al . Learning both weights and connections for efficient neural networks [EB/OL]. ( 2015-06-08 )[ 2021-12 ]. https://arxiv.org/abs/1506.02626 https://arxiv.org/abs/1506.02626 .

WEN W , WU C P , WANG Y D , et al . Learning structured sparsity in deep neural networks [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems . New York : ACM , 2016 : 2082 - 2090 .

LIU Z , LI J G , SHEN Z Q , et al . Learning efficient convolutional networks through network slimming [C]// 2017 IEEE International Conference on Computer Vision(ICCV) . Venice : IEEE , 2017 : 2755 - 2763 .

PARK E , YOO S , VAJDA P . Value-aware quantization for training and inference of neural networks [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 608 - 624 .

YAMAMOTO K . Learnable companding quantization for accurate low-bit neural networks [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 5027 - 5036 .

ZHANG D Q , YANG J L , YE D , et al . LQ-Nets: Learned quantization for highly accurate and compact deep neural networks [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 373 - 390 .

CHEN W L , WILSON J T , TYREE S , et al . Compressing neural networks with the hashing trick [C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning . New York : ACM , 2015 : 2285 - 2294 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [EB/OL]. ( 2015-03-09 )[ 2021-12 ]. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 .

ZHU J G , TANG S X , CHEN D P , et al . Complementary relation contrastive distillation [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 9256 - 9265 .

GOLUBEVA A , NEYSHABUR B , GUR-ARI G . Are wider nets better given the same number of parameters? [EB/OL]. ( 2020-10-27 )[ 2021-12 ]. https://arxiv.org/abs/2010.14495 https://arxiv.org/abs/2010.14495 .

袁海英 , 成君鹏 . 面向移动端图像分类的轻量级卷积神经网络的设计方法 : 202110462584.4 [P]. 2021-07-02 .

CHOLLET F . Xception: Deep learning with depthwise separable convolutions [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 1800 - 1807 .

HOWARD A , SANDLER M , CHEN B , et al . Searching for MobileNetV3 [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Seoul : IEEE , 2019 : 1314 - 1324 .

HAN K , WANG Y H , TIAN Q , et al . GhostNet: More features from cheap operations [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle : IEEE , 2020 : 1577 - 1586 .

HUANG G , LIU S C , MAATEN L V D , et al . CondenseNet: An efficient DenseNet using learned group convolutions [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 2752 - 2761 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于非一般类算子融合方法及硬件架构设计

MalMKNet：一种用于恶意代码分类的多尺度卷积神经网络

基于特征膨胀卷积模块的轻量化技术研究

一种轻量级的多尺度通道注意图像超分辨率重建网络