基于跨层注意力特征交互和多尺度通道注意力的单幅图像去雾网络

孙航; 付秋月; 李勃辉; 但志平; 余梅; 万俊

doi:10.12263/DZXB.20230821

您当前的位置：

首页 >

文章列表页 >

基于跨层注意力特征交互和多尺度通道注意力的单幅图像去雾网络

学术论文 | 更新时间：2025-12-11

- 基于跨层注意力特征交互和多尺度通道注意力的单幅图像去雾网络
- Cross-Layer Attention Feature Interaction and Multi-Scale Channel Attention Network for Single Image Dehazing
- 电子学报 2024年52卷第11期页码：3711-3726
- 作者机构：
  
  1.三峡大学计算机与信息学院，湖北宜昌 443000
  2.三峡大学湖北省水电工程智能视觉监测重点实验室，湖北宜昌 443000
  3.中南财经政法大学信息与安全工程学院，湖北武汉 430073
- 作者简介：
  
  孙航男，1986年4月出生于湖北省武汉市.武汉大学博士，现为三峡大学副教授，硕士生导师.主要研究方向为计算机视觉、图像去雾和水下图像修复. E-mail: sunhang0418@whu.edu.cn
  付秋月女，1998年8月出生于广东省佛山市.现为三峡大学硕士研究生.主要研究方向为计算机视觉、图像去雾. E-mail: 15272874521@163.com
  李勃辉男，1998年10月出生于河南省信阳市.现为三峡大学硕士研究生.主要研究方向为计算机视觉、图像去雾. E-mail： snowwhite@uestc.edu.cn
  但志平男，1976年3月出生于湖北省宜昌市.现为三峡大学教授.主要研究方向为模式识别、计算机视觉. E-mail: zp_dan@ctgu.edu.cn
  余梅女，1980年1月，湖北省枣阳市.武汉大学博士，现为三峡大学讲师.主要研究方向为计算机视觉、新型电力系统. E-mail: yumei@ctgu.edu.cn
- 基金信息：
  
  国家自然科学基金(62002233);湖北省自然科学基金(2021CFB004)
- DOI：10.12263/DZXB.20230821
  中图分类号： TP391.4
- 收稿：2023-08-29，
  
  修回：2023-12-27，
  
  纸质出版：2024-11-25
- 稿件说明：
移动端阅览
孙航, 付秋月, 李勃辉, 等. 基于跨层注意力特征交互和多尺度通道注意力的单幅图像去雾网络[J]. 电子学报, 2024, 52(11): 3711-3726.

SUN Hang, FU Qiu-yue, LI Bo-hui, et al. Cross-Layer Attention Feature Interaction and Multi-Scale Channel Attention Network for Single Image Dehazing[J]. Acta Electronica Sinica, 2024, 52(11): 3711-3726.
孙航, 付秋月, 李勃辉, 等. 基于跨层注意力特征交互和多尺度通道注意力的单幅图像去雾网络[J]. 电子学报, 2024, 52(11): 3711-3726. DOI：10.12263/DZXB.20230821

SUN Hang, FU Qiu-yue, LI Bo-hui, et al. Cross-Layer Attention Feature Interaction and Multi-Scale Channel Attention Network for Single Image Dehazing[J]. Acta Electronica Sinica, 2024, 52(11): 3711-3726. DOI：10.12263/DZXB.20230821

摘要

近年来，基于U型结构的卷积神经网络在去雾领域取得了显著的成果.然而，大多数基于U型结构的去雾网络将编码层特征直接传递到对应尺度的解码层，忽略了不同层次特征信息的有效利用.此外，去雾网络中广泛使用的通道注意力受感受野的限制，没有充分地利用上下文信息，从而对通道权重的学习起负面作用，使得重构的清晰图像不够理想.为了解决上述问题，本文提出了一种跨层注意力特征交互和多尺度通道注意力的去雾算法.具体来说，跨层注意力特征交互模块利用编码层的多尺度跨层特征学习层级权重，然后将这些跨层特征聚合传递到对应解码层，从而减少了去雾网络重构清晰图像过程中的特征稀释.此外，为了挖掘对于去雾网络非常重要的特征通道信息，本文设计了多尺度通道注意力机制，利用不同空洞率的空洞卷积提取多尺度特征信息，形成一个多尺度上下文并行学习的通道注意力机制，可以更有效地为去雾网络的特征分配权重.实验结果表明，本文提出的去雾算法在4个公开的数据集上相比现有的12种去雾方法取得了较好的客观评价指标和视

觉效果.本文的代码已上传至

https：//github.com/bohuisir/AAFMAN

https://github.com/bohuisir/AAFMAN

Abstract

In recent years

U-shaped convolutional neural networks (CNNs) have achieved remarkable progress in image dehazing. However

most U-shaped dehazing networks directly pass encoder features to the decoder at the corresponding scale

ignoring effective utilization of multi-scale features. In addition

channel attention widely used in dehazing networks is restricted by receptive fields

failing to sufficiently leverage contextual information

which adversely affects learning of channel weights. To address the above issues

this paper proposes a novel dehazing algorithm with cross-layer attentive feature interaction and multi-scale channel attention. Specifically

the cross-layer attentive feature interaction module learns hierarchical weights for multi-scale encoder features

and aggregates these cross-layer features for transfer to the decoder

thereby reducing feature dilution during the dehazing network's reconstruction of clear images. Moreover

to uncover channel information that is critical for dehazing networks

we devise a multi-scale channel attention mechanism that extracts multi-scale features by dilated convolutions with different dilation rates

forming a parallel learning scheme of channel attention with multi-scale contexts for more effective weight allocation for dehazing network features. Experimental results demonstrate that the proposed dehazing algorithm achieves better objective metrics and visual performance compared to 12 existing methods on 4 public datasets. The code for this paper has been uploaded to

http://github.com/bohuisir/AAFMAF

关键词

Keywords

references

KALWAR S , PATEL D , AANEGOLA A , et al . GDIP: Gated differentiable image processing for object detection in adverse conditions [C ] // 2023 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway : IEEE , 2023 : 7083 - 7089 .

张智 , 易华挥 , 郑锦 . 聚焦小目标的航拍图像目标检测算法 [J ] . 电子学报 , 2023 , 51 ( 4 ): 944 - 955 .

ZHANG Z , YI H H , ZHENG J . Focusing on small objects detector in aerial images [J ] . Acta Electronica Sinica , 2023 , 51 ( 4 ): 944 - 955 . (in Chinese)

LI S Y , FISCHER T , KE L , et al . OVTrack: Open-vocabulary multiple object tracking [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 5567 - 5577 .

CHEN X , PENG H W , WANG D , et al . SeqTrack: Sequence to sequence learning for visual object tracking [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 14572 - 14581 .

WAN J , XI H , ZHOU J , et al . Robust and precise facial landmark detection by self-calibrated pose attention network [J ] . IEEE Transactions on Cybernetics , 2023 , 53 ( 6 ): 3546 - 3560 .

YANG X , LIU C , XU L L , et al . Towards effective adversarial textured 3D meshes on physical face recognition [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 4119 - 4128 .

CANTOR A . Optics of the atmosphere: Scattering by molecules and particles [J ] . IEEE Journal of Quantum Electronics , 1978 , 14 ( 9 ): 698 - 699 .

BERMAN D , TREIBITZ T , AVIDAN S . Non-local image dehazing [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 1674 - 1682 .

HE K M , SUN J , TANG X O . Single image haze removal using dark channel prior [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 12 ): 2341 - 2353 .

CAI B L , XU X M , JIA K , et al . DehazeNet: An end-to-end system for single image haze removal [J ] . IEEE Transactions on Image Processing , 2016 , 25 ( 11 ): 5187 - 5198 .

ZHANG H , PATEL V M . Densely connected pyramid dehazing network [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 3194 - 3203 .

LI R D , PAN J S , HE M , et al . Task-oriented network for image dehazing [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 6523 - 6534 .

高继蕊 , 李华锋 , 张亚飞 , 等 . 双注意力引导的细节和结构信息融合图像去雾网络 [J ] . 电子学报 , 2023 , 51 ( 1 ): 160 - 171 .

GAO J R , LI H F , ZHANG Y F , et al . Dual attention-guided detail and structure information fusion network for image dehazing [J ] . Acta Electronica Sinica , 2023 , 51 ( 1 ): 160 - 171 . (in Chinese)

SUN H , LI B H , DAN Z P , et al . Multi-level feature interaction and efficient non-local information enhanced channel attention for image dehazing [J ] . Neural Networks , 2023 , 163 : 10 - 27 .

ZUNAIR H , BEN HAMZA A . Sharp U-Net: Depthwise convolutional network for biomedical image segmenta-tion [J ] . Computers in Biology and Medicine , 2021 , 136 : 104699 .

但志平 , 方帅领 , 孙航 , 等 . 基于双判别器异构CycleGAN框架下多阶通道注意力校准的室外图像去雾 [J ] . 电子学报 , 2023 , 51 ( 9 ): 2558 - 2571 .

DAN Z P , FANG S L , SUN H , et al . Outdoor image dehazing based on multi-order channel attention calibration using a dual-discriminator heterogeneous CycleGAN framework [J ] . Acta Electronica Sinica , 2023 , 51 ( 9 ): 2558 - 2571 . (in Chinese)

DONG H , PAN J S , XIANG L , et al . Multi-scale boosted dehazing network with dense feature fusion [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 2154 - 2164 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 2261 - 2269 .

WU H Y , QU Y Y , LIN S H , et al . Contrastive learning for compact single image dehazing [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogni-tion (CVPR) . Piscataway : IEEE , 2021 : 10546 - 10555 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

WEI X C , ZHANG Y , ZHENG Y H . BSFCDet: Bidirectional spatial-semantic fusion network coupled with channel attention for object detection in satellite images [J ] . Remote Sensing , 2023 , 15 ( 13 ): 3213 .

YANG X , LV Z Y , ATLI BENEDIKTSSON J , et al . Novel spatial-spectral channel attention neural network for land cover change detection with remote sensed images [J ] . Remote Sensing , 2022 , 15 ( 1 ): 87 .

LI H F , QIU K J , CHEN L , et al . SCAttNet: Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images [J ] . IEEE Geoscience and Remote Sensing Letters , 2021 , 18 ( 5 ): 905 - 909 .

ZHONG Z L , LIN Z Q , BIDART R , et al . Squeeze-and-attention networks for semantic segmentation [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 13062 - 13071 .

LI J , LIU K Y , HU Y T , et al . Eres-UNet++: Liver CT image segmentation based on high-efficiency channel attention and Res-UNet++ [J ] . Computers in Biology and Medicine , 2023 , 158 : 106501 .

QIN X , WANG Z L , BAI Y C , et al . FFA-net: Feature fusion attention network for single image dehazing [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 11908 - 11915 .

LI B Y , REN W Q , FU D P , et al . Benchmarking single image dehazing and beyond [J ] . IEEE Transactions on Image Processing , 2018 , 28 ( 1 ): 492 - 505 .

ANCUTI C O , ANCUTI C , SBERT M , et al . Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images [C ] // 2019 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2019 : 1014 - 1018 .

ANCUTI C O , ANCUTI C , TIMOFTE R . NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2020 : 1798 - 1805 .

HUANG B H , LI Z , YANG C , et al . Single satellite optical imagery dehazing using sar image prior based on conditional generative adversarial networks [C ] // 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2020 : 1795 - 1802 .

TANG K T , YANG J C , WANG J . Investigating haze-relevant features in a learning framework for image deha-zing [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2014 : 2995 - 3002 .

REN W Q , LIU S , ZHANG H , et al . Single image dehazing via multi-scale convolutional neural networks [[M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2016 : 154 - 169 .

LI Y N , MIAO Q G , OUYANG W L , et al . LAP-net: Level-aware progressive network for image dehazing [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 3275 - 3284 .

REN W Q , PAN J S , ZHANG H , et al . Single image dehazing via multi-scale convolutional neural networks with holistic edges [J ] . International Journal of Computer Vision , 2020 , 128 ( 1 ): 240 - 259 .

LI B Y , GOU Y B , GU S H , et al . You only look yourself: Unsupervised and untrained single image dehazing neural network [J ] . International Journal of Computer Vision , 2021 , 129 ( 5 ): 1754 - 1767 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 2242 - 2251 .

ENGIN D , GENC A , EKENEL H K . Cycle-dehaze: Enhanced CycleGAN for single image dehazing [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2018 : 938 - 946 .

ANVARI Z , ATHITSOS V . Dehaze-GLCGAN: Unpaired single image de-hazing via adversarial training [EB/OL ] . ( 2020-08-15 )[ 2023-03-29 ] . http://arxiv.org/abs/2008.06632v1 http://arxiv.org/abs/2008.06632v1 .

LI B Y , PENG X L , WANG Z Y , et al . AOD-net: All-in-one dehazing network [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 4780 - 4788 .

QU Y Y , CHEN Y Z , HUANG J Y , et al . Enhanced Pix2pix dehazing network [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 8152 - 8160 .

GUO H F , PIAO J C . MARG-UNet: A single image dehazing network based on multimodal attention resi-dual group [C ] // 2022 IEEE 2nd International Conference on Information Communication and Software Engineer-ing (ICICSE) . Piscataway : IEEE , 2022 : 105 - 109 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2018 : 3 - 19 .

WANG Q L , WU B G , ZHU P F , et al . ECA-net: Efficient channel attention for deep convolutional neural networks [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 11531 - 11539 .

CAI Y T , WANG Y . MA-Unet: An improved version of Unet based on multi-scale and attention mechanism for medical image segmentation [C ] // Third International Conference on Electronics and Communication; Network and Computer Technology (ECNCT 2021) . SPIE , 2022 : 1 - 13 .

ATES G C , MOHAN P , ÇELIK E . Dual cross-attention for medical image segmentation [J ] . Engineering Applications of Artificial Intelligence , 2023 , 126 : 107139 .

SHYAM P , YOON K J , KIM K S . Towards domain invariant single image dehazing [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 11 ): 9657 - 9665 .

LIN C Y , RONG X W , YU X Y . MSAFF-net: Multiscale attention feature fusion networks for single image dehazing and beyond [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 3089 - 3100 .

LEDIG C , THEIS L , HUSZÁR F , et al . Photo-realistic single image super-resolution using a generative adversarial network [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 105 - 114 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL ] . ( 2015-04-10 )[ 2023-03-29 ] . http://arxiv.org/abs/1409.1556v6 http://arxiv.org/abs/1409.1556v6 .

CHEN D D , HE M M , FAN Q N , et al . Gated context aggregation network for image dehazing and deraining [C ] // 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2019 : 1375 - 1383 .

YU Y K , LIU H , FU M H , et al . A two-branch neural network for non-homogeneous dehazing via ensemble learning [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2021 : 193 - 202 .

BAI H R , PAN J S , XIANG X G , et al . Self-guided image dehazing using progressive feature fusion [J ] . IEEE Transactions on Image Processing , 2022 , 31 : 1217 - 1229 .

WANG P Y , ZHU H Q , HUANG H , et al . TMS-GAN: A twofold multi-scale generative adversarial network for single image dehazing [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 5 ): 2760 - 2772 .

LIU Y Y , LIU H , LI L Y , et al . A data-centric solution to NonHomogeneous dehazing via vision transformer [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2023 : 1406 - 1415 .

MEI Y Q , FAN Y C , ZHOU Y Q . Image super-resolution with non-local sparse attention [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognit-ion (CVPR) . Piscataway : IEEE , 2021 : 3516 - 3525 .

AINSLIE J , ONTAÑÓN S , ALBERTI C , et al . ETC: Encoding long and structured inputs in transformers [EB/OL ] . ( 2020-10-27 )[ 2023-03-29 ] . http://arxiv.org/abs/2004.08483v5 http://arxiv.org/abs/2004.08483v5 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于梯度引导偏振度估算的图像去雾

基于双判别器异构CycleGAN框架下多阶通道注意力校准的室外图像去雾

基于DCNN和BiLSTM的单通道视听融合语音分离方法研究

双注意力引导的细节和结构信息融合图像去雾网络