Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net

YUAN Hai-ying; CHENG Jun-peng; ZENG Zhi-yong; WU Yan-rui

doi:10.12263/DZXB.20211671

您当前的位置：

首页 >

文章列表页 >

Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net

PAPERS | 更新时间：2025-07-02

- Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net
- ACTA ELECTRONICA SINICA Vol. 51, Issue 1, Pages: 180-191(2023)
- 作者机构：
  
  北京工业大学信息学部，北京 100124
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20211671
  CLC： TP391;
- Received：18 December 2021，
  
  Revised：2022-03-07，
  
  Published：25 January 2023
- 稿件说明：
移动端阅览
袁海英,成君鹏,曾智勇等.Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J].电子学报,2023,51(01):180-191.

YUAN Hai-ying,CHENG Jun-peng,ZENG Zhi-yong,et al.Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net[J].ACTA ELECTRONICA SINICA,2023,51(01):180-191.
袁海英,成君鹏,曾智勇等.Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J].电子学报,2023,51(01):180-191. DOI： 10.12263/DZXB.20211671.

YUAN Hai-ying,CHENG Jun-peng,ZENG Zhi-yong,et al.Mobile_BLNet: Optimization Design of Lightweight Convolutional Neural Network Based on Big-Little Net[J].ACTA ELECTRONICA SINICA,2023,51(01):180-191. DOI： 10.12263/DZXB.20211671.

摘要

针对深度卷积神经网络难以部署到资源受限的端侧设备这一问题，本文提出一种高效精简的轻量化卷积神经网络Mobile_BLNet，在模型规模、计算量和性能之间取得了良好的平衡.该网络引入深度可分离卷积和倒残差结构，通过合理分配不同分支的运算量缩减模型规模并节省大量计算资源；采用通道剪枝操作压缩网络模型，基于占总和比值方法裁剪对模型贡献度低的卷积通道，在相同压缩效果情况下提升了分类准确率；基于通道裁剪情况重构网络，进一步降低模型所需计算资源.实验结果表明，Mobile_BLNet结构精简、性能优异，在CIFAR-10/CIFAR-100数据集上以0.1 M/0.3 M参数量、9.6 M/12.7 M浮点计算量获得91.2%/71.5%分类准确率；在Food101/ImageNet数据集上以1.0 M/2.1 M参数量、203.0 M/249.6 M浮点计算量获得82.8%/70.9%分类准确率，满足轻量化卷积神经网络的端侧硬件高能效部署需求.

Abstract

Since it is difficult for deep convolutional neural network to be deployed to terminal equipment with limited resources

this paper proposes an efficient

compact

and lightweight network Mobile_BLNet

which achieves a good balance between model size

computation

and performance. The network uses depthwise separable convolution and inverse residual structure

reduces the scale of the model and saves a lot of computing resources by reasonably allocating the amount of computation of different branches. The total ratio method is used to prune the convolution channel with low contribution

which has excellent performance under the same compression effect. Model reconstruction is based on the clipping

which further reduces the computational resources. The experimental results show that Mobile_BLNet has excellent performance. On CIFAR-10/CIFAR-100 dataset

91.2%/71.5% accuracy is obtained with 0.1 M/0.3 M parameters and 9.6 M/12.7 M floating point operations. On Food101/ImageNet dataset

82.8%/70.9% accuracy is obtained with 1.0 M/2.1 M parameters and 203.0 M/249.6 M floating point operations. The network meets the requirements of energy-efficient and lightweight hardware deployment.

关键词

Keywords

references

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [C]// Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe Nevada : ACM , 2012 : 1097 - 1105 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL]. ( 2014-09-04 )[ 2021-12 ]. https://arxiv.org/abs/1409.1556 https://arxiv.org/abs/1409.1556 .

SZEGEDY C , LIU W , JIA Y Q , et al . Going deeper with convolutions [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston : IEEE , 2015 : 1 - 9 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 770 - 778 .

HUANG H H , ZHOU P , LI Y , et al . A lightweight attention-based CNN model for efficient gait recognition with wearable IMU sensors [J]. Sensors(Basel) , 2021 , 21 ( 8 ): 2866 .

SHUVO S B , ALI S N , SWAPNIL S I , et al . A lightweight CNN model for detecting respiratory diseases from lung auscultation sounds using EMD-CWT-based hybrid scalogram [J]. IEEE Journal of Biomedical and Health Informatics , 2021 , 25 ( 7 ): 2595 - 2603 .

IANDOLA F N , HAN S , MOSKEWICZ M W , et al . SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0 . 5 MB model size[EB/OL]. ( 2016-02-24 )[ 2021-12 ]. https://arxiv.org/abs/1602.07360 https://arxiv.org/abs/1602.07360 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. ( 2017-04-17 )[ 2021-12 ]. https://arxiv.org/abs/1704.04861 https://arxiv.org/abs/1704.04861 .

SANDLER M , HOWARD A , ZHU M L , et al . MobileNetV2: Inverted residuals and linear bottlenecks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 4510 - 4520 .

ZHANG X Y , ZHOU X Y , LIN M X , et al . ShuffleNet: An extremely efficient convolutional neural network for mobile devices [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 6848 - 6856 .

MA N N , ZHANG X Y , ZHENG H T , et al . ShuffleNet V2: Practical guidelines for efficient CNN architecture design [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 122 - 138 .

TAN M X , LE Q V . EfficientNet: Rethinking model scaling for convolutional neural networks [C]// Proceedings of the 36th International Conference on Machine Learning . Long Beach : MLResearch Press , 2019 , 97 : 6105 - 6114 .

CHEN C F , FAN Q F , MALLINAR N , et al . Big-Little Net: An efficient multi-scale feature representation for visual and speech recognition [EB/OL]. ( 2018-07-10 )[ 2021-12 ]. https://arxiv.org/abs/1807.03848 https://arxiv.org/abs/1807.03848 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 2261 - 2269 .

DENTON E , ZAREMBA W , BRUNA J , et al . Exploiting linear structure within convolutional networks for efficient evaluation [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems . New York : ACM , 2014 : 1269 - 1277 .

DONG X Y , HUANG J S , YANG Y , et al . More is less: A more complicated network with less inference complexity [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 1895 - 1903 .

LI H , KADAV A , DURDANOVIC I , et al . Pruning filters for efficient ConvNets [EB/OL]. ( 2016-08-31 )[ 2021-12 ]. https://arxiv.org/abs/1608.08710 https://arxiv.org/abs/1608.08710 .

TANG Y H , WANG Y H , XU Y X , et al . Manifold regularized dynamic network pruning [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 5016 - 5026 .

HAN S , POOL J , TRAN J , et al . Learning both weights and connections for efficient neural networks [EB/OL]. ( 2015-06-08 )[ 2021-12 ]. https://arxiv.org/abs/1506.02626 https://arxiv.org/abs/1506.02626 .

WEN W , WU C P , WANG Y D , et al . Learning structured sparsity in deep neural networks [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems . New York : ACM , 2016 : 2082 - 2090 .

LIU Z , LI J G , SHEN Z Q , et al . Learning efficient convolutional networks through network slimming [C]// 2017 IEEE International Conference on Computer Vision(ICCV) . Venice : IEEE , 2017 : 2755 - 2763 .

PARK E , YOO S , VAJDA P . Value-aware quantization for training and inference of neural networks [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 608 - 624 .

YAMAMOTO K . Learnable companding quantization for accurate low-bit neural networks [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 5027 - 5036 .

ZHANG D Q , YANG J L , YE D , et al . LQ-Nets: Learned quantization for highly accurate and compact deep neural networks [C]// European Conference on Computer Vision . Munich : Springer , 2018 : 373 - 390 .

CHEN W L , WILSON J T , TYREE S , et al . Compressing neural networks with the hashing trick [C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning . New York : ACM , 2015 : 2285 - 2294 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [EB/OL]. ( 2015-03-09 )[ 2021-12 ]. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 .

ZHU J G , TANG S X , CHEN D P , et al . Complementary relation contrastive distillation [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 9256 - 9265 .

GOLUBEVA A , NEYSHABUR B , GUR-ARI G . Are wider nets better given the same number of parameters? [EB/OL]. ( 2020-10-27 )[ 2021-12 ]. https://arxiv.org/abs/2010.14495 https://arxiv.org/abs/2010.14495 .

袁海英 , 成君鹏 . 面向移动端图像分类的轻量级卷积神经网络的设计方法 : 202110462584.4 [P]. 2021-07-02 .

CHOLLET F . Xception: Deep learning with depthwise separable convolutions [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 1800 - 1807 .

HOWARD A , SANDLER M , CHEN B , et al . Searching for MobileNetV3 [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Seoul : IEEE , 2019 : 1314 - 1324 .

HAN K , WANG Y H , TIAN Q , et al . GhostNet: More features from cheap operations [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle : IEEE , 2020 : 1577 - 1586 .

HUANG G , LIU S C , MAATEN L V D , et al . CondenseNet: An efficient DenseNet using learned group convolutions [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 2752 - 2761 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

MalMKNet: A Multi-Scale Convolutional Neural Network Used for Malware Classification

Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution

Image Super-Resolution Reconstruction Based on Lightweight Multi-Scale Channel Attention Network

Related Author

YUAN Hai-ying

CHENG Jun-peng

ZENG Zhi-yong

WU Yan-rui

WANG Ying

GAO Lan

ZHANG Zhe

LIU Xin

Related Institution

Faculty of Information Technology， Beijing University of Technology

College of Information Engineering, Capital Normal University

School of Mathematical Science, Capital Normal University

Faculty of Software Technologics, Shanxi Agricultural University

Institute of Air Defense and Anti-missile， Air Force Engineering University

⁰