基于多分支结构的不确定性局部通道注意力机制

doi:10.12263/DZXB.20201204

电子学报 ›› 2022, Vol. 50 ›› Issue (2): 374-382.DOI: 10.12263/DZXB.20201204

基于多分支结构的不确定性局部通道注意力机制

伍邦谷¹, 张苏林², 石红¹, 朱鹏飞¹, 王旗龙¹, 胡清华¹

^1.天津大学智能与计算学部, 天津 300350
^2.中汽数据(天津)有限公司, 天津 300380

收稿日期:2020-10-28 修回日期:2021-01-25 出版日期:2022-02-25
- 通讯作者:
- 王旗龙
- 作者简介:
- 伍邦谷男，1996年生于湖北省.硕士研究生，计算机科学与技术专业，研究方向为计算机视觉. E-mail: wubanggu@tju.edu.cn
  张苏林男，1987年生于辽宁省.中汽数据（天津）有限公司高级技术经理，主要研究方向为智能网联汽车、自动驾驶控制及感知系统. E-mail: zhangsulin@catarc.ac.cn
  石红女，1975年生于内蒙古自治区.副教授，硕士生导师.主要研究方向为粗糙集、机器学习. E-mail: serena@tju.edu.cn
  朱鹏飞男，1986年生于河南省.副教授，博士生导师.主要研究方向为无人机视觉、人机协同学习、度量学习. E-mail: zhupengfei@tju.edu.cn
  王旗龙（通讯作者）男，1989年生于黑龙江省.副教授，硕士生导师.主要研究方向为视频图像分析、深度学习等. E-mail: qlwang@tju.edu.cn
  胡清华男，1976年生于湖南省.教授，博士生导师.主要研究方向为数据不确定性建模、多模态数据学习. E-mail: huqinghua@tju.edu.cn
- 基金资助:
- 国家自然科学基金 (61806140)

Multi-Branch Structure Based Local Channel Attention with Uncertainty

WU Bang-gu¹, ZHANG Su-lin², SHI Hong¹, ZHU Peng-fei¹, WANG Qi-long¹, HU Qing-hua¹

^1.College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
^2.Automotive Data of China Co., Ltd., Tianjin 300380, China

Received:2020-10-28 Revised:2021-01-25 Online:2022-02-25 Published:2022-02-25
- Corresponding author:
- WANG Qi-long
- Supported by:
- National Natural Science Foundation of China (61806140)

摘要/Abstract

摘要：

近几年的研究表明视觉注意力机制是提升深层卷积神经网络性能的有效途径.然而，现有的视觉注意力方法更多地致力于建模所有卷积通道之间的相关性，在一定程度上限制了模型的计算效率.此外，这些方法尚未明确考虑相关性建模过程中不确定性带来的影响，缺少对注意力机制在泛化能力和稳定性方面的探索.为解决上述问题，提出了一种多分支局部通道注意力模块（Multi-Branch Local Channel Attention，MBLCA）.通过建模通道之间的局部相关性学习各个通道的权重，提升了模型的计算效率.并采用蒙特卡洛（Monte Carlo，MC）Dropout近似的深度贝叶斯学习方法对局部通道注意力模块进行不确定性建模，从而得到一个多分支的局部通道注意力模块.提出的MBLCA模块可以灵活地应用于各种深层卷积神经网络架构中，与同类型的工作相比，嵌入MBLCA模块的ResNet-50网络结构在ImageNet-1K和MS COCO数据集上分别取得了2.58%的分类精度提升和1.9%的AP提升.

关键词: 通道注意力机制, 不确定性, 多分支结构, 深层卷积神经网络

Abstract:

Recent researches demonstrate that attention mechanism is an effective way to improve performance of deep convolution neural networks(CNNs). However, most of existing attention methods more dedicate to modeling the correlation between all channels, which limits the computational efficiency of the model. In addition, these methods have not considered the impact of uncertainty in the correlation modeling process, and lack the exploration of the generalization ability and stability of the attention mechanism. A multi-branch local channel attention(MBLCA) module is proposed to handle above issues. MBLCA learns channel attention by capturing correlation across channels in a local range instead of global ones, improving the computational efficiency, and models the uncertainty of local channel attention by deep Bayesian learning, which is approximated by Monte Carlo(MC) Dropout, leading a multi-branch structure. The proposed MBLCA can be flexibly adopted to various deep CNN architectures. For example, ResNet-50 with the MBLCA module has achieved 2.58% improvement in classification accuracy and 1.9% improvement in average precise on the ImageNet-1K and MS COCO datasets against state-of-the-art counterparts.

Key words: channel attention mechanism, uncertainty, multi-branch structure, deep convolutional neural networks

中图分类号:

TP183

伍邦谷, 张苏林, 石红, 等. 基于多分支结构的不确定性局部通道注意力机制[J]. 电子学报, 2022, 50(2): 374-382.

Bang-gu WU, Su-lin ZHANG, Hong SHI, et al. Multi-Branch Structure Based Local Channel Attention with Uncertainty[J]. Acta Electronica Sinica, 2022, 50(2): 374-382.

图/表 7

图1 多分支局部通道注意力（MBLCA）模块整体框图

表1 不同注意力机制方法的模型复杂度分析

模型	MBLCA	ECA	SE	CBAM
参数量	$k$	$k$	$2 C 2 r$	$2 C 2 r + 49 ⋅ 2$
计算量	$k n C ⋅ B$	$k C ⋅ B$	$2 C 2 ⋅ B r$	$2 C 2 ⋅ B r + H ⋅ W ⋅ 49 ⋅ 2$

表1 不同注意力机制方法的模型复杂度分析

模型	MBLCA	ECA	SE	CBAM
参数量	$k$	$k$	$2 C 2 r$	$2 C 2 r + 49 ⋅ 2$
计算量	$k n C ⋅ B$	$k C ⋅ B$	$2 C 2 ⋅ B r$	$2 C 2 ⋅ B r + H ⋅ W ⋅ 49 ⋅ 2$

图2 MBLCA模块对于不同分支数量（n）和Dropout概率（p）组合的消融实验

图3 MBLCA模块对于卷积核大小k的消融实验

表2 各种当前最优的视觉注意力方法的分类结果

方法	骨干网络	参数量	计算量	Top-1	Top-5
ResNet	ResNet-18	11.15 M	1.70 G	70.40	89.45
SENet		11.23 M	1.70 G	70.59	89.78
CBAM		11.23 M	1.70 G	70.73	89.91
ECA-Net		11.15 M	1.70 G	70.78_±0.12	89.92_±0.07
MBLCA		11.15 M	1.70 G	70.88_±0.06	89.89_±0.03
ResNet	ResNet-50	24.37 M	3.86 G	75.20	92.52
SENet		26.77 M	3.87 G	76.71	93.38
CBAM		26.77 M	3.87 G	77.34	93.69
A2-Net		33.00 M	6.50 G	77.00	93.50
AA-Net		25.80 M	4.15 G	77.70	93.80
GSoP-Net1		28.05 M	6.18 G	77.68	93.98
ECA-Net		24.37 M	3.86 G	77.48_±0.25	93.68_±0.15
MBLCA		24.37 M	3.86 G	77.78_±0.12	93.89_±0.07
ResNet	ResNet-101	42.49 M	7.34 G	76.83	93.48
SENet		47.01 M	7.35 G	77.62	93.93
CBAM		47.01 M	7.35 G	78.49	94.31
AA-Net		45.40 M	8.05 G	78.70	94.40
ECA-Net		42.49 M	7.35 G	78.65_±0.23	94.34_±0.15
MBLCA		42.49 M	7.35 G	78.85_±0.11	94.48_±0.08
ResNet-200		74.45 M	14.10 G	78.20	94.00
Inception-v3		25.90 M	5.36 G	77.45	93.56
DenseNet-161		27.35 M	7.34 G	77.65	93.80

表3 各种方法在MS COCO2017验证集上的目标检测和实例分割结果

方法	参数量	计算量	目标检测						实例分割
方法	参数量	计算量	AP	AP₅₀	AP₇₅	AP_S	AP_M	AP_L	AP	AP₅₀	AP₇₅	AP_S	AP_M	AP_L
ResNet-50	44.18 M	275.58 G	37.2	58.9	40.3	22.2	40.7	48.0	34.1	55.5	36.2	16.1	36.7	50.0
+SE block	46.67 M	275.69 G	38.7	60.9	42.1	23.4	42.7	50.0	35.4	57.4	37.8	17.1	38.6	51.8
+ECA	44.18 M	275.69 G	39.0	61.3	42.1	24.2	42.8	49.9	35.6	58.1	37.7	17.6	39.0	51.8
+MBLCA	44.18 M	275.69 G	39.1	61.2	42.4	23.3	42.8	50.1	35.5	57.7	37.5	17.0	38.6	51.9
ResNet-101	63.17 M	351.65 G	39.4	60.9	43.3	23.0	43.7	51.4	35.9	57.7	38.4	16.8	39.1	53.6
+SE block	67.89 M	351.84 G	40.7	62.5	44.3	23.9	45.2	52.8	36.8	59.3	39.2	17.2	40.3	53.6
+ECA	63.17 M	351.83 G	41.3	63.1	44.8	25.1	45.8	52.9	37.4	59.9	39.8	18.1	41.1	54.1
+MBLCA	63.17 M	351.83 G	41.4	63.4	45.2	24.8	45.9	53.3	37.4	59.9	39.9	18.0	41.0	54.3

表4 各种方法在ImageNet-C数据集上测试结果

方法	错误率↓	mCE↓	Noise↓	Blur↓	Weather↓	Digital↓
ResNet50	24.80	76.70	81.67	80.50	69.00	77.50
+SE block	23.29	72.80	73.33	78.00	66.75	73.25
+ECA	22.52	72.07	73.67	77.75	67.50	69.75
+MBLCA	22.22	71.20	71.33	76.25	66.00	71.25
ResNet101	23.17	70.30	75.33	73.75	64.50	69.00
+SE block	22.38	69.13	70.33	75.00	62.50	69.00
+ECA	21.35	67.07	64.67	73.00	63.25	66.75
+MBLCA	21.15	66.53	64.33	73.25	62.25	65.75

参考文献 39

1	YOSHUAB, AARONC, PASCALV. Representation learning: A review and new perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828.
2	KRIZHEVSKYA, SUTSKEVERI, HINTONG E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe, Ne: MIT Press, 2012: 1097-1105.
3	DENGJ, DONGW, SOCHERR, LIL, LIK, LIF F. ImageNet: A large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA: IEEE, 2009: 248-255.
4	SIMONYANK, ZISSERMANA. Very deep convolutional networks for large-scale image recognition[J]. arXiv:, 2015.
5	SZEGEDYC, LIUW, JIAY, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 1-9. DOI: 10.1109/CVPR.2015.7298594.
6	HEK, ZHANGX, RENS, SUNJ. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 770-778.
7	HUANGG, LIUZ, MAATENL, WEINBERGERK Q. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 2261-2269.
8	HUJ, SHENL, SUNG. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 18-23.
9	WOO S, PARKJ, LEEJ, KWEONI S. CBAM: Convolutional block attention module[C]//European Conference on Computer Vision. Munich, Germany: Springer, 2018: 3-19.
10	BELLOI, ZOPHB, VASWANIA, SHLENSJ, LEQ. Attention augmented convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul, Korea (South): IEEE, 2019: 3285-3294.
11	CHENY, KALANTIDISY, LIJ, YANS, FENGJ. A²-Nets: Double attention networks[C]//Advances In Neural Information Processing Systems. Montreal, Canada: MIT Press, 2018: 352-361.
12	FUJ, LIUJ, TIANH, LIY, BAOY, FANGZ, LUH. Dual attention network for scene segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 3146-3154.
13	GAOZ, XIEJ, WANGQ, LIP. Global second-order pooling convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 3024-3033.
14	WANGX, GIRSHICKR, GUPTAA, HEK. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 7794-7803.
15	WANGQ, WUB, ZHUP, LIP, ZUOW, HUQ. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA: IEEE, 2020: 11531-11539.
16	J C MACKAYDAVID. A practical Bayesian framework for backpropagation networks[J]. Neural Computation. 1992, 4(3): 448-472.
17	GRAVESA. Practical variational inference for neural networks[C]//Advances in Neural Information Processing Systems. Red Hook, USA: MIT Press, 2011: 2348-2356.
18	GAL Y, GHAHRAMANIZ. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning[C]//International Conference on Machine Learning. New York, NY, USA: ACM, 2016: 1050-1059.
19	SRIVASTAVAN, HINTONG E, KRIZHEVSKYA, SUTSKEVERI, SALAKHUTDINOVR. Dropout: A simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
20	LINT, MAIREM, BELONGIES J, HAYSJ, PERONAP, RAMANAND, DOLL'ARP, ZITNICKC L. Microsoft COCO: Common objects in context[C]//European Conference on Computer Vision. Munich, Germany: Springer, 2014: 8693: 740-755.
21	仇祝令, 查宇飞, 吴敏, 等. 基于注意力学习的正则化相关滤波跟踪算法[J]. 电子学报, 2020, 48(9): 1762-1768.
	QIUZ L, ZHAY F, WUM, et al. Learning attentional regularized correlation filter for visual tracking[J]. Acta Electronica Sinica, 2020, 48(9): 1762-1768. (in Chinese)
22	QINP, SHENW, ZENGJ. DSCA-Net: Indoor head detection network using dual-stream information and channel attention[J]. Chinese Journal of Electronics, 2020, 29(6): 1102-1109.
23	盖杉, 王俊生. 基于深度学习的非局部注意力增强网络图像去雨算法研究[J]. 电子学报, 2020, 48(10): 1899-1908.
	GAIS, WANGJ S. Image raindrop algorithm research using nonlocal attention enhanced network based on deep learning[J]. Acta Electronica Sinica, 2020, 48(10): 1899-1908. (in Chinese)
24	CAOY, XUJ, LINS, WEIF, HUH. GCnet: Non-local networks meet squeeze-excitation networks and beyond[C]//International Conference on Computer Vision Workshops. Seoul, Korea(South): IEEE, 2019: 1971-1980.
25	PAISLEYJ W, BLEID M, JORDANM I. Variational Bayesian inference with stochastic search[C]//International Conference on Machine Learning. Edinburgh: ACM, 2012.
26	KINGMAD P, WELLINGM. Auto-encoding variational Bayes[C]//International Conference on Learning Representations. Banff, AB, Canada: IEEE, 2014: 1-14.
27	HOFFMANM D, LEI D MB, WANGC, PAISLEYJ W. Stochastic variational inference[J]. The Journal of Machine Learning Research. 2013, 14(1): 1303-1347.
28	齐现英, 刘伯强, 徐建伟. 基于不确定性信息融合的高密度椒盐噪声降噪方法[J]. 电子学报, 2016, 44(4): 878-885.
	QIX Y, LIUB Q, XUJ W. A novel algorithm for removing high-density salt-and-pepper noise based on fusion of indeterminacy information[J]. Acta Electronica Sinica, 2016, 44(4): 878-885. (in Chinese)
29	张挺, 刘金华. 一种新的空间数据不确定性重建方法[J]. 电子学报, 2018, 46(3): 641-645.
	ZHANGT, LIUJ H. A New indefinite reconstruction method for spatial data[J]. Acta Electronica Sinica, 2018, 46(3): 641-645. (in Chinese)
30	TAKASHIY, SHINGOM, KOTAROH, et al. Recurrent neural networks with multi-branch structure[J]. IEEJ Transactions on Electronics Information & Systems, 2007, 127(9): 1430-1435.
31	CHENY, LIJ, XIAOH, JINX, YANS, FENGJ. Dual path networks[C]//Advances in Neural Information Processing Systems. Long Beach: MIT Press, 2017: 4467-4475.
32	NAIRV, HINTONG E. Rectified linear units improve restricted Boltzmann machines[C]//International Conference on Machine Learning. Haifa: ACM, 2010: 807-814.
33	HEK, GKIOXARIG, DOLL'ARP, GIRSHICKR B. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision(ICCV). Venice, Italy: IEEE. 2017: 2980-2988.
34	DANH, THOMASG D. Benchmarking neural network robustness to common corruptions and perturbations[C]//International Conference on Learning Representation. New Orleans, LA, USA: IEEE, 2019.
35	LINT, DOLL'ARP, GIRSHICKR B, HEK, HARIHARANB, BELONGIES J. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 936-944.
36	HEK, ZHANGX, RENS, SUNJ. Identity mappings in deep residual networks[C]//European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016: 630-645.
37	SZEGEDYC, VANHOUCKEV, IOFFES, SHLENSJ, WOJNAZ. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 2818-2826.
38	SANDLERM, HOWARDA G, ZHUM, ZHMOGINOVA, CHENL. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 4510-4520.
39	SZEGEDYC, IOFFES, VANHOUCKEV, ALEMIA A. Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI Press, 2017: 4278-4284.

[1]	王硕, 王坚, 王亚男, 宋亚飞. 一种基于特征融合的恶意代码快速检测方法[J]. 电子学报, 2023, 51(1): 57-66.
[2]	张清华, 黄志康, 高满, 艾志华. 基于不确定性与错误分类率博弈的序贯三支决策模型[J]. 电子学报, 2022, 50(5): 1033-1041.
[3]	张浩, 胡昌华, 杜党波, 裴洪, 张建勋. 多状态影响下基于Bi‑LSTM网络的锂电池剩余寿命预测方法[J]. 电子学报, 2022, 50(3): 619-624.
[4]	智慧来, 张丽, 李金海. 旁观者视角下粒的多层次描述[J]. 电子学报, 2022, 50(11): 2568-2574.
[5]	陶凌, 陈安庆, 邓贞宙, 韩春雷, 王玉皞, 王平. 应用分形几何与混沌理论进行植物形态建模[J]. 电子学报, 2021, 49(9): 1776-1782.
[6]	张志勇, 程曦. 色散有损介质土壤对电磁回波影响的不确定性分析研究[J]. 电子学报, 2021, 49(3): 614-618.
[7]	潘刚, 尚朝轩, 梁玉英, 蔡金燕, 孟亚峰. 相关竞争失效场合雷达功率放大系统可靠性评估[J]. 电子学报, 2017, 45(4): 805-812.
[8]	张清华, 薛玉斌, 胡峰, 于洪. 粗糙集近似集不确定性研究[J]. 电子学报, 2016, 44(7): 1574-1580.
[9]	齐现英, 刘伯强, 徐建伟. 基于不确定性信息融合的高密度椒盐噪声降噪方法[J]. 电子学报, 2016, 44(4): 878-885.
[10]	张龙昌, 杨艳红, 赵绪辉. 基于云模型的SaaS决策方法[J]. 电子学报, 2015, 43(5): 987-992.
[11]	李泓波, 张健沛, 杨静, 白劲波, 初妍. 基于拓扑势的重叠社区及社区间结构洞识别——兼论结构洞理论视角下网络的脆弱性[J]. 电子学报, 2014, 42(1): 62-69.
[12]	朱婷婷, 王丽娜, 胡东辉, 付建伟, 王旻杰. 基于不确定性推理的JPEG图像通用隐藏信息检测技术[J]. 电子学报, 2013, 41(2): 233-238.
[13]	杨丽, 徐扬. 基于逻辑合取的语言真值概念格合并算法[J]. 电子学报, 2013, 41(11): 2149-2155.
[14]	张健沛, 李泓波, 杨静, 白劲波, 张乐君, 初妍. 基于归属不确定性的变规模网络重叠社区识别[J]. 电子学报, 2012, 40(12): 2512-2518.
[15]	王永利;钱江波;孙淑荣;张功萱;刘冬梅. AMUR:一种RFID数据不确定性的自适应度量算法[J]. 电子学报, 2011, 39(3): 579-584.

基于多分支结构的不确定性局部通道注意力机制

Multi-Branch Structure Based Local Channel Attention with Uncertainty

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 39

相关文章 15

编辑推荐

Metrics