基于深度学习的非局部注意力增强网络图像去雨算法研究

doi:10.3969/j.issn.0372-2112.2020.10.004

电子学报 ›› 2020, Vol. 48 ›› Issue (10): 1899-1908.DOI: 10.3969/j.issn.0372-2112.2020.10.004

基于深度学习的非局部注意力增强网络图像去雨算法研究

盖杉, 王俊生

南昌航空大学信息工程学院, 江西南昌 330063

收稿日期:2019-08-09 修回日期:2020-06-04 出版日期:2020-10-25 发布日期:2020-10-25
作者简介:盖杉男,1980年3月出生,吉林长春人,博士,副教授,主要研究方向为人工智能、模式识别与机器学习.E-mail:gaishan@nchu.edu.cn
王俊生 男,1992年10月出生,安徽芜湖人,硕士研究生,主要研究方向为机器学习与模式识别.
基金资助:
国家自然科学基金（No.61563037）；江西省杰出青年基金（No.20192ACB21032）

Image Raindrop Algorithm Research Using Nonlocal Attention Enhanced Network Based on Deep Learning

GAI Shan, WANG Jun-sheng

School of Information Engineering, Nanchang Hangkong University, Nanchang, Jiangxi 330063, China

Received:2019-08-09 Revised:2020-06-04 Online:2020-10-25 Published:2020-10-25

摘要/Abstract

摘要： 单幅图像去雨技术的瓶颈问题是在缺少帧与帧时间序列信息的情况下，如何能够在有效去除多密度雨条纹的同时保留图像背景的细节结构信息.针对该问题，本文提出了一种新的基于编码解码器结构的单幅图像去雨算法.首先利用非局部操作获得不同像素点间的位置关系信息，从而获得图像全局信息表征.其次，采用空间注意力机制对全局信息在空间维度位置上进行权值重标定，即在通道维度上对特征进行非线性建模，从而达到聚集相似特征和有用信息的目的.最后，利用反卷积与长距离残差连接逐层恢复去雨图像的大小.分析和实验结果表明，本文提出算法雨痕去除效果明显，有效解决了去除具有不同雨密度大小雨条纹的现实困难，同时较好地保留图像的细节和边缘信息.

关键词: 注意力机制, 非局部, 有益信息, 反卷积, 边缘信息

Abstract: The bottleneck problem of single image de-raining is how to preserve the detailed structure information of image background with a lack of time series data between frames when removing the multi-density rain fringe.The new single image de-raining algorithm based on the coder and decoder structure is proposed in this paper.Firstly,the positional relationship information of various pixels between the points is obtained by non-local operation which can obtain the global information of the image representation.Secondly,the spatial attention is applied to recalibrate the global information in the spatial dimension position,and the channel features are nonlinearly modeled in the channel dimension to aggregate similar characteristics and useful information.Finally,the original size of rainy image is obtained by utilizing the de-convolution and long distance residual connection.The experimental results and analysis show that the proposed algorithm can obtain significant de-raindrop effect.The proposed algorithm can also resolve the difficulties of removing raindrops with various densities while maintaining the details of the image and edge information.

Key words: attention mechanism, non-local, useful information, de-convolution, edge information

中图分类号:

TP751
TP183

盖杉, 王俊生. 基于深度学习的非局部注意力增强网络图像去雨算法研究[J]. 电子学报, 2020, 48(10): 1899-1908.

GAI Shan, WANG Jun-sheng. Image Raindrop Algorithm Research Using Nonlocal Attention Enhanced Network Based on Deep Learning[J]. Acta Electronica Sinica, 2020, 48(10): 1899-1908.

参考文献

[1] Kang L W,Lin C W,Fu Y H.Automatic single-image-based rain streaks removal via image decomposition[J].IEEE Transactions on Image Processing,2012,21(4):1742-1755.
[2] Li Y,Robby T,Guo X J,et al.Rain streak removal using layer priors[J].IEEE Transactions on Image Processing,2017,26(8):3874-3885.
[3] Luo Y,Xu Y,Ji H.Removing rain from a single image via discriminative sparse coding[A].Proceedings of the IEEE International Conference on Computer Vision[C].Santiago:IEEE,2015.10-18.
[4] Chang Y,Yan L X,Zhong S.Transformed low-rank model for line pattern noise removal[A].Proceedings of the IEEE International Conference on Computer Vision[C].Venice:IEEE,2017.1-9.
[5] Fu X Y,Huang J B,Ding X H,et al.Clearing the skies:A deep network architecture for single-image rain removal[J].IEEE Transactions on Image Processing,2017,26(6):2944-2956.
[6] Fu X Y,Huang J B,Zeng D L,et al.Removing rain from single images via a deep detail network[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2017.1715-1723.
[7] Zhang H,Patel V M.Density-aware single image deraining using a multi-stream dense network[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.695-704.
[8] Li X,Wu J L,Lin Z C,et al.Recurrent squeeze-and-excitation context aggregation net for single image deraining[A].European Conference on Computer Vision[C].Berlin:Springer,2018.262-277.
[9] Zhang H,Sindagi V,Patel V M.Image deraining using a conditional generative adversarial network[J].IEEE Transactions on Circuits and Systems for Video Technology,2019,1(1):1-12.
[10] Yang W H,Robby T,Feng J S,et al.Deep joint rain detection and removal from a single image[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2017.1357-1366.
[11] Pan J S,Liu S F,Sun D Q,et al.Learning dual convolutional neural networks for low-level vision[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.3070-3079.
[12] 李康,李亚敏,胡学敏,等.基于卷积神经网络的鲁棒高精度目标跟踪算法[J].电子学报,2018,46(9):2087-2093. LI Kang,LI Ya-min,HU Xue-min,et al.A robust and accurate object tracking algorithm based on convolutional neural network[J].Acta Electronica Sinica,2018,46(9):2087-2093.(in Chinese)
[13] Wang X L,Girshick R,Gupta A,et al.Non-local neural networks[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.16-25.
[14] Woo S,Park J,Lee J Y,et al.CBAM:Convolutional block attention module[A].European Conference on Computer Vision[C].Berlin:Springer,2018.1-17.
[15] Chen L C,Zhu Y K,George P,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[A].European Conference on Computer Vision[C].Berlin:Springer,2018.160-168.
[16] Hu J,Shen L,Sun G.Squeeze-and-excitation networks[A].Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.18-23.

基于深度学习的非局部注意力增强网络图像去雨算法研究

Image Raindrop Algorithm Research Using Nonlocal Attention Enhanced Network Based on Deep Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王延达, 陈炜通, 皮德常, 岳琳. 一种自适应记忆神经网络多跳读取与覆盖度机制结合的药物推荐模型[J]. 电子学报, 2022, 50(4): 943-953.
[2]	张凌明, 赵悦, 李鹏程, 刘洋, 高陈强. 基于局部注意力机制的三维牙齿模型分割网络[J]. 电子学报, 2022, 50(3): 681-690.
[3]	伍邦谷, 张苏林, 石红, 朱鹏飞, 王旗龙, 胡清华. 基于多分支结构的不确定性局部通道注意力机制[J]. 电子学报, 2022, 50(2): 374-382.
[4]	王波, 黄冕, 刘利军, 黄青松, 单文琦. 基于多层聚焦Inception-V3卷积网络的细粒度图像分类[J]. 电子学报, 2022, 50(1): 72-78.
[5]	赵琰, 赵凌君, 匡纲要. 基于注意力机制特征融合网络的SAR图像飞机目标快速检测[J]. 电子学报, 2021, 49(9): 1665-1674.
[6]	周东明, 张灿龙, 李志欣, 王智文. 基于多层级视觉融合的图像描述模型[J]. 电子学报, 2021, 49(7): 1286-1290.
[7]	李志欣, 孙亚茹, 唐素勤, 张灿龙, 马慧芳. 双路注意力引导图卷积网络的关系抽取[J]. 电子学报, 2021, 49(2): 315-323.
[8]	江泽涛, 钱艺, 伍旭, 张少钦. 一种基于ARD‑GAN的低照度图像增强方法[J]. 电子学报, 2021, 49(11): 2160-2165.
[9]	石义乐, 杨文忠, 杜慧祥, 王丽花, 王婷, 理珊珊. 基于深度学习的图像描述综述[J]. 电子学报, 2021, 49(10): 2048-2060.
[10]	唐海桃, 薛嘉宾, 韩纪庆. 一种多尺度前向注意力模型的语音识别方法[J]. 电子学报, 2020, 48(7): 1255-1260.
[11]	张志昌, 曾扬扬, 庞雅丽. 融合语义角色和自注意力机制的中文文本蕴含识别[J]. 电子学报, 2020, 48(11): 2162-2169.
[12]	尤洪峰, 田生伟, 禹龙, 吕亚龙. 基于Word Embedding的遥感影像检测分割[J]. 电子学报, 2020, 48(1): 75-83.
[13]	冯建周, 宋沙沙, 王元卓, 刘亚坤, 武红颖, 龚昊. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700.
[14]	蒋明峰, 陆亮, 吴龙, 徐文龙, 汪亚明. 基于加权Schatten p范数最小化的磁共振图像重构方法研究[J]. 电子学报, 2019, 47(4): 784-790.
[15]	孙忠贵, 高新波, 张冬梅, 李洁, 王颖. 一个具有附益性及保序性算子的形态学非局部拓展[J]. 电子学报, 2018, 46(8): 1969-1975.