电子学报 ›› 2023, Vol. 51 ›› Issue (1): 160-171.DOI: 10.12263/DZXB.20211549
高继蕊1,2, 李华锋1,2, 张亚飞1,2, 谢明鸿1, 李凡1,2
收稿日期:
2021-11-19
修回日期:
2022-04-25
出版日期:
2023-01-25
作者简介:
基金资助:
GAO Ji-rui1,2, LI Hua-feng1,2, ZHANG Ya-fei1,2, XIE Ming-hong1, LI Fan1,2
Received:
2021-11-19
Revised:
2022-04-25
Online:
2023-01-25
Published:
2023-02-23
Supported by:
摘要:
雾图像结构信息弱化、边缘细节信息丢失,严重影响其在高水平视觉任务的使用.现有大部分去雾方法对图像细节信息的恢复并不理想,影响了图像去雾的整体效果.为此,本文提出一种双注意力引导的细节和结构信息融合去雾网络.该网络主要由空间-通道双注意力联合模块、细节和结构信息融合模块以及多尺度特征重建模块组成.其中,空间-通道双注意力联合模块通过联合空间和通道两个维度的注意力进行特征提取,实现雾图像中细节和结构信息的增强;细节和结构信息融合模块将结构信息和边缘细节信息融合为注意力权重和逆向注意力权重,以进一步增强这两种信息;多尺度特征重建模块将提取到的特征重建为清晰图像.实验结果表明,本文方法的去雾效果在定量评价和视觉效果上均优于对比方法.
中图分类号:
高继蕊, 李华锋, 张亚飞, 等. 双注意力引导的细节和结构信息融合图像去雾网络[J]. 电子学报, 2023, 51(1): 160-171.
Ji-rui GAO, Hua-feng LI, Ya-fei ZHANG, et al. Dual Attention-Guided Detail and Structure Information Fusion Network for Image Dehazing[J]. Acta Electronica Sinica, 2023, 51(1): 160-171.
方法 | SOTS-outdoor | HSTS-Syn | HAZERD |
---|---|---|---|
CAP(TIP’15)[ | 18.12/0.758 1/0.135 | 19.71/0.809 5/0.114 | 14.12/0.708 2/0.267 |
DehazeNet(TIP’16)[ | 22.61/0.863 3/0.075 | 23.92/0.888 0/0.065 | 15.52/0.760 1/0.236 |
EPDN(CVPR’19)[ | 19.96/0.834 2/0.144 | 21.35/0.883 3/0.098 | 15.70/0.754 7/0.267 |
FFA-Net(AAAI’20)[ | 30.89/ | 31.36/ | 16.26/0.791 0/0.253 |
DA(CVPR’20)[ | 25.79/0.886 4/0.197 | 26.26/0.864 7/0.170 | 17.65/ |
TBNN(CVPRW’21)[ | 15.21/0.780 6/0.186 | 13.65/0.747 9/0.186 | 12.03/0.745 7/0.293 |
AOD-Net(ICCV’17)[ | 19.65/0.854 8/0.097 | 19.77/0.839 4/0.099 | 15.45/0.749 3/0.269 |
DPSID(ITM’18)[ | 18.44/0.861 7/0.119 | 18.45/0.849 5/0.120 | 15.52/0.772 4/0.224 |
本文方法 |
表1 不同方法在不同测试集上去雾结果的定量评价结果(PSNR/SSIM/LPIPS)
方法 | SOTS-outdoor | HSTS-Syn | HAZERD |
---|---|---|---|
CAP(TIP’15)[ | 18.12/0.758 1/0.135 | 19.71/0.809 5/0.114 | 14.12/0.708 2/0.267 |
DehazeNet(TIP’16)[ | 22.61/0.863 3/0.075 | 23.92/0.888 0/0.065 | 15.52/0.760 1/0.236 |
EPDN(CVPR’19)[ | 19.96/0.834 2/0.144 | 21.35/0.883 3/0.098 | 15.70/0.754 7/0.267 |
FFA-Net(AAAI’20)[ | 30.89/ | 31.36/ | 16.26/0.791 0/0.253 |
DA(CVPR’20)[ | 25.79/0.886 4/0.197 | 26.26/0.864 7/0.170 | 17.65/ |
TBNN(CVPRW’21)[ | 15.21/0.780 6/0.186 | 13.65/0.747 9/0.186 | 12.03/0.745 7/0.293 |
AOD-Net(ICCV’17)[ | 19.65/0.854 8/0.097 | 19.77/0.839 4/0.099 | 15.45/0.749 3/0.269 |
DPSID(ITM’18)[ | 18.44/0.861 7/0.119 | 18.45/0.849 5/0.120 | 15.52/0.772 4/0.224 |
本文方法 |
方法 | PSNR | SSIM | LPIPS |
---|---|---|---|
BL | 26.50 | 0.954 9 | 0.022 4 |
BL+S-C | 27.90 | 0.962 8 | 0.017 7 |
BL+S-C+DS | 28.75 | 0.963 3 | 0.016 5 |
BL+S-C+DS+MR | 29.43 | 0.968 5 | 0.014 3 |
BL+CP+DS+MR | 27.70 | 0.957 8 | 0.019 7 |
W/O | 29.03 | 0.966 2 | 0.016 7 |
W/O | 28.05 | 0.962 1 | 0.018 1 |
W/O MFEB | 28.80 | 0.965 3 | 0.015 5 |
表2 本文提出的方法中不同模块消融实验的结果
方法 | PSNR | SSIM | LPIPS |
---|---|---|---|
BL | 26.50 | 0.954 9 | 0.022 4 |
BL+S-C | 27.90 | 0.962 8 | 0.017 7 |
BL+S-C+DS | 28.75 | 0.963 3 | 0.016 5 |
BL+S-C+DS+MR | 29.43 | 0.968 5 | 0.014 3 |
BL+CP+DS+MR | 27.70 | 0.957 8 | 0.019 7 |
W/O | 29.03 | 0.966 2 | 0.016 7 |
W/O | 28.05 | 0.962 1 | 0.018 1 |
W/O MFEB | 28.80 | 0.965 3 | 0.015 5 |
方法 | Model size/MB | Params/M | FLOPs/G |
---|---|---|---|
CAP | — | — | — |
DehazeNet | 0.030 | 0.008 | 0.394 |
AOD-Net | 0.009 | 0.002 | 0.088 |
EPDN | 66.300 | 17.379 | 3.677 |
DA | 208.354 | 54.591 | 30.041 |
FFA-Net | 25.390 | 4.456 | 220.551 |
TBNN | 192.769 | 50.352 | 64.736 |
DPSID | — | — | — |
Ours | 5.410 | 1.342 | 71.562 |
表3 不同方法的模型大小、参数量和计算量大小比较
方法 | Model size/MB | Params/M | FLOPs/G |
---|---|---|---|
CAP | — | — | — |
DehazeNet | 0.030 | 0.008 | 0.394 |
AOD-Net | 0.009 | 0.002 | 0.088 |
EPDN | 66.300 | 17.379 | 3.677 |
DA | 208.354 | 54.591 | 30.041 |
FFA-Net | 25.390 | 4.456 | 220.551 |
TBNN | 192.769 | 50.352 | 64.736 |
DPSID | — | — | — |
Ours | 5.410 | 1.342 | 71.562 |
1 | PANG Y W, LI Y Z, SHEN J B, et al. Towards bridging semantic gap to improve semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4229-4238. |
2 | MA S, PANG Y, PAN J, et al. Preserving details in semantics-aware context for scene parsing[J]. Science China Information Sciences, 2020, 63(2): 120106. |
3 | ZHANG Z J, PANG Y W. CGNet: Cross-guidance network for semantic segmentation[J]. Science China Information Sciences, 2020, 63(2): 120104. |
4 | NIE J, ANWER R M, CHOLAKKAL H, et al. Enriched feature guided refinement network for object detection[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9536-9545. |
5 | PANG Y W, XIE J, KHAN M H, et al. Mask-guided attention network for occluded pedestrian detection[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, : 4966-4974. |
6 | CAO J L, PANG Y W, HAN J G, et al. Hierarchical shot detector[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9704-9713. |
7 | LI Y Z, PANG Y W, SHEN J B, et al. NETNet: Neighbor erasing and transferring network for better single shot object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13346-13355. |
8 | NI W P, GAO X B, WANG Y. Single satellite image dehazing via linear intensity transformation and local property analysis[J]. Neurocomputing, 2016, 175: 25-39. |
9 | MCCARTNEY E J, HALL F F. Optics of the atmosphere: Scattering by molecules and particles[J]. Physics Today, 1977, 30(5): 76-77. |
10 | NARASIMHAN S G, NAYAR S K. Chromatic framework for vision in bad weather[C]//Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 598-605. |
11 | NARASIMHAN S G, NAYAR S K. Vision and the atmosphere[J]. International Journal of Computer Vision, 2002, 48(3): 233-254. |
12 | HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. |
13 | ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3522-3533. |
14 | BERMAN D, TREIBITZ T, AVIDAN S. Non-local image dehazing[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1674-1682. |
15 | CAI B L, XU X M, JIA K, et al. DehazeNet: An end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198. |
16 | REN W, LIU S, ZHANG H, et al. Single image dehazing via multi-scale convolutional neural networks[C]//European Conference on Computer Vision. Amsterdam: Springer Cham, 2016: 154-169. |
17 | ZHANG H, PATEL V M. Densely connected pyramid dehazing network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3194-3203. |
18 | LI Y N, MIAO Q G, OUYANG W L, et al. LAP-Net: Level-aware progressive network for image dehazing[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3275-3284. |
19 | REN W Q, MA L, ZHANG J W, et al. Gated fusion network for single image dehazing[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3253-3261. |
20 | LIU X H, MA Y R, SHI Z H, et al. GridDehazeNet: Attention-based multi-scale network for image dehazing[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7314-7323. |
21 | QU Y Y, CHEN Y Z, HUANG J Y, et al. Enhanced pix2pix dehazing network[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, : 8152-8160. |
22 | QIN X, WANG Z L, BAI Y C, et al. FFA-Net: Feature fusion attention network for single image dehazing[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI Press, 2020: 11908-11915. |
23 | DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2154-2164. |
24 | ZHANG X Q, JIANG R H, WANG T, et al. Single image dehazing via dual-path recurrent network[J]. IEEE Transactions on Image Processing, 2021, 30: 5211-5222. |
25 | 张登银, 鞠铭烨, 王雪梅. 一种基于暗通道先验的快速图像去雾算法[J]. 电子学报, 2015, 43(7): 1437-1443. |
ZHANG D Y, JU M Y, WANG X M. A fast image daze removal algorithm using dark channel prior[J]. Acta Electronica Sinica, 2015, 43(7): 1437-1443. (in Chinese) | |
26 | 刘杰平, 黄炳坤, 韦岗. 一种快速的单幅图像去雾算法[J]. 电子学报, 2017, 45(8): 1896-1901. |
LIU J P, HUANG B K, WEI G. A fast effective single image dehazing algorithm[J]. Acta Electronica Sinica, 2017, 45(8): 1896-1901. (in Chinese) | |
27 | FATTAL R. Dehazing using color-lines[J]. ACM Transactions on Graphics, 2014, 34(1): 13. |
28 | 肖进胜, 周景龙, 雷俊锋, 等. 基于霾层学习的单幅图像去雾算法[J]. 电子学报, 2019, 47(10): 2142-2148. |
XIAO J S, ZHOU J L, LEI J F, et al. Single image dehazing algorithm based on the learning of hazy layers[J]. Acta Electronica Sinica, 2019, 47(10): 2142-2148. (in Chinese) | |
29 | SHAO Y J, LI L, REN W Q, et al. Domain adaptation for image dehazing[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2805-2814. |
30 | YU Y K, LIU H, FU M H, et al. A two-branch neural network for non-homogeneous dehazing via ensemble learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2021: 193-202. |
31 | MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal: MIT Press, 2014: 2204-2212. |
32 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. |
33 | Fan D P, Ji G P, Zhou T, et al. PraNet: Parallel reverse attention network for polyp segmentation[C]//International Conference on Medical Image Computing and Computer Assisted Intervention. Lima: Springer, 2020: 263-273. |
34 | LI S L, ZHAO M, FANG Z Y, et al. Image super-resolution using lightweight multiscale residual dense network[J]. International Journal of Optics, 2020, 2020: 2852865. |
35 | Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2021-11-10]]. . |
36 | WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. |
37 | LI B Y, REN W Q, FU D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. |
38 | ZHANG Y F, DING L, SHARMA G. HazeRD: An outdoor scene dataset and benchmark for single image dehazing[C]//2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3205-3209. |
39 | Mannos J, Sakrison D. The effects of a visual fidelity criterion of the encoding of images[J]. IEEE Transactions on Information Theory, 1974, 20(4): 525-536. |
40 | ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595. |
41 | Kingma D P, Ba J. Adam: A method for stochastic optimization[EB/OL].(2014-10-22)[2021-11-10]. . |
42 | HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 558-567. |
43 | LI B Y, PENG X L, WANG Z Y, et al. AOD-net: All-in-one dehazing network[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4780-4788. |
44 | GAO Y Y, HU H M, LI B, et al. Detail preserved single image dehazing algorithm based on airlight refinement[J]. IEEE Transactions on Multimedia, 2019, 21(2): 351-362. |
[1] | 隗昊, 唐焕玲, 周爱, 张益嘉, 陈飞, 鲁明羽. 基于双路分段注意力神经张量网络的临床文本关系抽取[J]. 电子学报, 2023, 51(3): 658-665. |
[2] | 贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245. |
[3] | 何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56. |
[4] | 王硕, 王坚, 王亚男, 宋亚飞. 一种基于特征融合的恶意代码快速检测方法[J]. 电子学报, 2023, 51(1): 57-66. |
[5] | 丁琪, 田萱, 孙国栋. 基于注意力增强的热点感知新闻推荐模型[J]. 电子学报, 2023, 51(1): 93-104. |
[6] | 金天虎, 陶砚蕴, 李佐勇. 基于超像素图像分割的暗通道先验去雾改进算法[J]. 电子学报, 2023, 51(1): 146-159. |
[7] | 梁顺攀, 刘伟, 郑智中, 原福永. 基于方面和胶囊网络的跨域评分预测模型[J]. 电子学报, 2022, 50(9): 2102-2109. |
[8] | 刘群, 谭洪胜, 张优敏, 王国胤. 基于元路径的动态异质网络表示学习[J]. 电子学报, 2022, 50(8): 1830-1839. |
[9] | 张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN:用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858. |
[10] | 任莎莎, 刘琼. 小目标特征增强图像分割算法[J]. 电子学报, 2022, 50(8): 1894-1904. |
[11] | 赵俊男, 佘青山, 孟明, 陈云. 基于多流空间注意力图卷积SRU网络的骨架动作识别[J]. 电子学报, 2022, 50(7): 1579-1585. |
[12] | 麻文刚, 张亚东, 郭进. 基于景深先验引导与环境光优化的图像去雾[J]. 电子学报, 2022, 50(7): 1708-1721. |
[13] | 李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435. |
[14] | 严莉, 张凯, 徐浩, 韩圣亚, 刘珅岐, 史玉良. 基于图注意力机制和Transformer的异常检测[J]. 电子学报, 2022, 50(4): 900-908. |
[15] | 王延达, 陈炜通, 皮德常, 岳琳. 一种自适应记忆神经网络多跳读取与覆盖度机制结合的药物推荐模型[J]. 电子学报, 2022, 50(4): 943-953. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||