基于深度学习的单幅图像去雾研究进展

贾童瑶; 卓力; 李嘉锋; 张菁

doi:10.12263/DZXB.20220838

您当前的位置：

首页 >

文章列表页 >

基于深度学习的单幅图像去雾研究进展

综述评论 | 更新时间：2025-07-02

- 基于深度学习的单幅图像去雾研究进展
- Research Advances on Deep Learning Based Single Image Dehazing
- 电子学报 2023年51卷第1期页码：231-245
- 作者机构：
  
  1.北京工业大学信息学部，北京 100124
  2.北京工业大学计算智能与智能系统北京市重点实验室，北京 100124
- 作者简介：
  
  [ "贾童瑶女.北京工业大学博士生.研究方向为图像复原/增强.E-mail: jty@emails.bjut.edu.cn" ]
  [ "卓力（通讯作者）女.博士，北京工业大学教授、博士生导师.主要研究方向为图像/视频的编码与传输、多媒体大数据处理等." ]
  [ "李嘉锋男.博士，北京工业大学副教授.主要研究方向为图像恢复/增强、目标检测和目标跟踪等.E-mail: lijiafeng@bjut.edu.cn" ]
  [ "张菁女.博士，北京工业大学教授、博士生导师.主要研究方向为图像处理、图像识别、图像检索等.E-mail: zhj@bjut.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61871006;61971016);北京市教育委员会科学研究计划项目资助(KZ202210005007;KZ201910005007);北京市自然科学基金-丰台轨道交通前沿研究联合基金(L211017);北京市教育委员会科技计划一般项目(KM202110005027)
- DOI：10.12263/DZXB.20220838
  中图分类号： TP391;
- 收稿：2022-07-17，
  
  修回：2022-10-06，
  
  纸质出版：2023-01-25
- 稿件说明：
移动端阅览
贾童瑶,卓力,李嘉锋等.基于深度学习的单幅图像去雾研究进展[J].电子学报,2023,51(01):231-245.

JIA Tong-yao,ZHUO Li,LI Jia-feng,et al.Research Advances on Deep Learning Based Single Image Dehazing[J].ACTA ELECTRONICA SINICA,2023,51(01):231-245.
贾童瑶,卓力,李嘉锋等.基于深度学习的单幅图像去雾研究进展[J].电子学报,2023,51(01):231-245. DOI： 10.12263/DZXB.20220838.

JIA Tong-yao,ZHUO Li,LI Jia-feng,et al.Research Advances on Deep Learning Based Single Image Dehazing[J].ACTA ELECTRONICA SINICA,2023,51(01):231-245. DOI： 10.12263/DZXB.20220838.

摘要

户外视觉系统极易受到雾霾等恶劣天气影响，采集到的图像/视频质量严重下降，这不仅影响人眼的主观感受，也给后续的智能化分析带来严峻挑战.近年来，学者们将深度学习应用于图像去雾领域，取得了诸多的研究成果.但是雾霾图像场景复杂多变、降质因素众多，这对去雾算法的泛化能力提出了很高的要求.本文主要总结了近年来基于深度学习的单幅图像去雾技术研究进展.从先验知识和物理模型、映射关系建模、数据样本、知识迁移学习等角度出发，介绍了现有算法的研究思路、具体特点、优势与不足.尤其侧重于近两年来新出现的训练策略和网络结构，如元学习、小样本学习、域自适应、Transformer等.另外，本文在公共数据集上对比了各种代表性去雾算法的主客观性能、模型复杂度等，尤其是分析了去雾后的图像对于后续目标检测任务的影响，更全面地评价了现有算法性能的优劣，并探讨了未来可能的研究方向.

Abstract

Vision-based outdoor systems are highly susceptible to severe weather such as haze. The quality of the collected images in the hazy environments is seriously degraded

which affects subjective perception and brings challenges to the subsequent intelligent processing tasks. In recent years

deep learning has been applied to single image dehazing and achieved promising results. However

the hazy scenes are complex and unpredictable

which puts a high demand on the generalization ability of the dehazing methods. In this paper

we summarize the recent deep-learning-based single-image dehazing methods. The advantages and disadvantages of these methods are analyzed in terms of network mapping relationships

learning methods

training datasets

and knowledge transfer. In particular

we focus on new training strategies and network structures that have emerged in the last few years

such as meta-learning

few-shot learning

domain adaption

and Transformer. In addition

the subjective and objective performances of various representative dehazing methods are compared on several public datasets. Further

the impact of the dehazed images on the performance of subsequent object detection tasks is analyzed and evaluated comprehensively. We also provide the computational complexity and running time of these methods. Finally

the conclusions and future tendency of single-image dehazing are drawn.

关键词

Keywords

references

LI J F , ZHUO L , ZHANG H , et al . Effective data-driven technology for efficient vision-based outdoor industrial systems [J]. IEEE Transactions on Industrial Informatics , 2020 , 16 ( 7 ): 4344 - 4354 .

郭璠 , 蔡自兴 , 谢斌 , 等 . 图像去雾技术研究综述与展望 [J]. 计算机应用 , 2010 , 30 ( 9 ): 2417 - 2421 .

GUO F , CAI Z X , XIE B , et al . Review and prospect of image dehazing techniques [J]. Journal of Computer Applications , 2010 , 30 ( 9 ): 2417 - 2421 . (in Chinese)

沈琛 , 曹风云 , 杨雪洁 . 夜间图像去雾研究综述与展望 [J]. 电子测量与仪器学报 , 2020 , 34 ( 11 ): 101 - 114 .

SHEN C , CAO F Y , YANG X J . Review and prospect of nighttime haze removal [J]. Journal of Electronic Measurement and Instrumentation , 2020 , 34 ( 11 ): 101 - 114 . (in Chinese)

郑凤仙 , 王夏黎 , 何丹丹 , 等 . 单幅图像去雾算法研究综述 [J]. 计算机工程与应用 , 2022 , 58 ( 3 ): 1 - 14 .

ZHENG F X , WANG X L , HE D D , et al . Survey of single image defogging algorithm [J]. Computer Engineering and Applications , 2022 , 58 ( 3 ): 1 - 14 . (in Chinese)

HE K M , SUN J , TANG X O . Single image haze removal using dark channel prior [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 12 ): 2341 - 2353 .

ZHU Q S , MAI J M , SHAO L . A fast single image haze removal algorithm using color attenuation prior [J]. IEEE Transactions on Image Processing , 2015 , 24 ( 11 ): 3522 - 3533 .

BERMAN D , TREIBITZ T , AVIDAN S . Non-local image dehazing [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1674 - 1682 .

MCCARTNEY E J , HALL F F . Optics of the atmosphere: Scattering by molecules and particles [J]. Physics Today , 1977 , 30 ( 5 ): 76 - 77 .

NAYAR S K , NARASIMHAN S G . Vision in bad weather [C]// Proceedings of the Seventh IEEE International Conference on Computer Vision . Kerkyra : IEEE , 1999 : 820 - 827 .

NARASIMHAN S G , NAYAR S K . Removing weather effects from monochrome images [C]// Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Kauai : IEEE , 2001 : II .

JU M Y , DING C , REN W Q , et al . IDE: Image dehazing and exposure using an enhanced atmospheric scattering model [J]. IEEE Transactions on Image Processing , 2021 , 30 : 2180 - 2192 .

JU M Y , DING C , GUO C A , et al . IDRLP: Image dehazing using region line prior [J]. IEEE Transactions on Image Processing , 2021 , 30 : 9043 - 9057 .

CHEN W T , DING J J , KUO S Y . PMS-Net: Robust haze removal based on patch map for single images [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 11673 - 11681 .

CAI B L , XU X M , JIA K , et al . DehazeNet: An end-to-end system for single image haze removal [J]. IEEE Transactions on Image Processing , 2016 , 25 ( 11 ): 5187 - 5198 .

LI B Y , PENG X L , WANG Z Y , et al . AOD-Net: All-in-one dehazing network [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 4780 - 4788 .

ZHANG J , TAO D C . FAMED-net: A fast and accurate multi-scale end-to-end dehazing network [J]. IEEE Transactions on Image Processing , 2020 , 29 : 72 - 84 .

ZHANG H , PATEL V M . Densely connected pyramid dehazing network [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 3194 - 3203 .

LI Y N , MIAO Q G , OUYANG W L , et al . LAP-Net: Level-aware progressive network for image dehazing [C]// 2019 IEEE/CVF International Conference on Computer Vision . Seoul : IEEE , 2019 : 3275 - 3284 .

LIU Y , PAN J S , REN J , et al . Learning deep priors for image dehazing [C]// 2019 IEEE/CVF International Conference on Computer Vision . Seoul : IEEE , 2019 : 2492 - 2500 .

LI R D , PAN J S , HE M , et al . Task-oriented network for image dehazing [J]. IEEE Transactions on Image Processing , 2020 , 29 : 6523 - 6534 .

BAI H R , PAN J S , XIANG X G , et al . Self-guided image dehazing using progressive feature fusion [J]. IEEE Transactions on Image Processing , 2022 , 31 : 1217 - 1229 .

BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: A deep convolutional encoder-decoder architecture for image segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 12 ): 2481 - 2495 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J]. Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

RADFORD A , METZ L , CHINTALA S , et al . Unsupervised representation learning with deep convolutional generative adversarial networks [C]// The 4th International Conference on Learning Representations . San Juan : International Conference on Learning Representations , 2016 : 149803 .

MIRZA M , OSINDERO S . Conditional generative adversarial nets . ( 2014-11-06 )[ 2022-07 ]. https://arxiv.org/abs/1411.1784 https://arxiv.org/abs/1411.1784 .

ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein GAN . ( 2017-01-26 )[ 2022-07 ]. https://arxiv.org/abs/1701.07875 https://arxiv.org/abs/1701.07875 .

LI R D , PAN J S , LI Z C , et al . Single image dehazing via conditional generative adversarial network [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 8202 - 8211 .

REN W Q , MA L , ZHANG J W , et al . Gated fusion network for single image dehazing [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 3253 - 3261 .

QU Y Y , CHEN Y Z , HUANG J Y , et al . Enhanced Pix2pix dehazing network [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 8152 - 8160 .

LIU X , SUGANUMA M , SUN Z , et al . Dual residual networks leveraging the potential of paired operations for image restoration [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 7000 - 7009 .

LIU X H , MA Y R , SHI Z H , et al . GridDehazeNet: attention-based multi-scale network for image dehazing [C]// 2019 IEEE/CVF International Conference on Computer Vision . Seoul : IEEE , 2019 : 7313 - 7322 .

DONG H , PAN J S , XIANG L , et al . Multi-scale boosted dehazing network with dense feature fusion [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 2154 - 2164 .

QIN X , WANG Z L , BAI Y C , et al . FFA-net: Feature fusion attention network for single image dehazing [C]// The Thirty-Fourth AAAI Conference on Artificial Intelligence . New York : AAAI Press , 2020 , 34 ( 7 ): 11908 - 11915 .

WU H Y , QU Y Y , LIN S H , et al . Contrastive learning for compact single image dehazing [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE , 2021 : 10546 - 10555 .

ZHENG Z R , REN W Q , CAO X C , et al . Ultra-high-definition image dehazing via multi-guided bilateral learning [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE , 2021 : 16180 - 16189 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C]// 31st Annual Conference on Neural Information Processing Systems . Long Beach : Curran Associates Inc. , 2017 : 6000 - 6010 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16 x 16 words: Transformers for image recognition at scale[EB/OL]. ( 2020-10-22 )[ 2022-07 ]. https://arxiv.org/abs/2010.11929 https://arxiv.org/abs/2010.11929 .

CHEN H T , WANG Y H , GUO T Y , et al . Pre-trained image processing transformer [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE , 2021 : 12294 - 12305 .

TOLSTIKHIN I , HOULSBY N , KOLESNIKOV A , et al . MLP-mixer: An all-MLP architecture for vision [EB/OL]. ( 2021-05-04 )[ 2022-07 ]. https://arxiv.org/abs/2105.01601 https://arxiv.org/abs/2105.01601 .

LIU H , DAI Z , SO D R , et al . Pay attention to MLPs [C]// Advances in Neural Information Processing Systems . Virtual Conference : NeurIPS , 2021 , 34 : 9204 - 9215 .

4141］ TOUVRON H , BOJANOWSKI P , CARON M , et al . ResMLP: Feedforward networks for image classification with data-efficient training [EB/OL]. ( 2021-05-07 )[ 2022-07 ]. https://arxiv.org/abs/2105.03404 https://arxiv.org/abs/2105.03404 .

TU Z Z , TALEBI H , ZHANG H , et al . MAXIM: multi-axis MLP for image processing [C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans : IEEE , 2022 : 5759 - 5770 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 2242 - 2251 .

YI Z L , ZHANG H , TAN P , et al . DualGAN: Unsupervised dual learning for image-to-image translation [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 2868 - 2876 .

KIM T , CHA M , KIM H , et al . Learning to discover cross-domain relations with generative adversarial networks [C]// Proceedings of the 34th International Conference on Machine Learning . Sydney : JMLR.org , 2017 : 1857 - 1865 .

ENGIN D , GENC A , EKENEL H K . Cycle-dehaze: Enhanced CycleGAN for single image dehazing [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City : IEEE , 2018 : 938 - 9388 .

LI Y N , LIU Y H , YAN Q X , et al . Deep dehazing network with latent ensembling architecture and adversarial learning [J]. IEEE Transactions on Image Processing , 2021 , 30 : 1354 - 1368 .

LI J F , LI Y P , ZHUO L , et al . USID-Net: Unsupervised single image dehazing network via disentangled representations [J/OL]. IEEE Transactions on Multimedia . DOI: 10.1109/TMM.2022.3163554 http://dx.doi.org/10.1109/TMM.2022.3163554 .

ZHAO S Y , ZHANG L , SHEN Y , et al . RefineDNet: A weakly supervised refinement framework for single image dehazing [J]. IEEE Transactions on Image Processing , 2021 , 30 : 3391 - 3404 .

LI B Y , GOU Y B , LIU J Z , et al . Zero-shot image dehazing [J]. IEEE Transactions on Image Processing , 2020 , 29 : 8457 - 8466 .

KAR A , DHARA S K , SEN D , et al . Zero-shot single image restoration through controlled perturbation of koschmieder's model [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE , 2021 : 16200 - 16210 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [EB/OL]. ( 2015-03-09 )[ 2022-07 ]. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 .

HONG M , XIE Y , LI C H , et al . Distilling image dehazing with heterogeneous task imitation [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 3459 - 3468 .

JIA T Y , LI J F , ZHUO L , et al . Effective meta-attention dehazing networks for vision-based outdoor industrial systems [J]. IEEE Transactions on Industrial Informatics , 2022 , 18 ( 3 ): 1511 - 1520 .

LI L , DONG Y L , REN W Q , et al . Semi-supervised image dehazing [J]. IEEE Transactions on Image Processing , 2020 , 29 : 2766 - 2779 .

SHAO Y J , LI L , REN W Q , et al . Domain adaptation for image dehazing [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 2805 - 2814 .

ZHANG K S , LI Y N . Single image dehazing via semi-supervised domain translation and architecture search [J]. IEEE Signal Processing Letters , 2021 , 28 : 2127 - 2131 .

CHEN Z Y , WANG Y C , YANG Y , et al . PSD: principled synthetic-to-real dehazing guided by physical priors [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE , 2021 : 7176 - 7185 .

ZHANG Y F , DING L , SHARMA G . HazeRD: An outdoor scene dataset and benchmark for single image dehazing [C]// 2017 IEEE International Conference on Image Processing . Beijing : IEEE , 2017 : 3205 - 3209 .

ANCUTI C , ANCUTI C O , TIMOFTE R , et al . I-HAZE: A dehazing benchmark with real hazy and haze-free indoor images [C]// International Conference on Advanced Concepts for Intelligent Vision Systems . Poitiers : Springer , 2018 : 620 - 631 .

ANCUTI C O , ANCUTI C , TIMOFTE R , et al . O-HAZE: A dehazing benchmark with real hazy and haze-free outdoor images [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City : IEEE , 2018 : 867 - 8678 .

ANCUTI C O , ANCUTI C , SBERT M , et al . Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images [C]// 2019 IEEE International Conference on Image Processing . Taipei : IEEE , 2019 : 1014 - 1018 .

ANCUTI C O , ANCUTI C , TIMOFTE R . NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Seattle : IEEE , 2020 : 1798 - 1805 .

LI B Y , REN W Q , FU D P , et al . Benchmarking single-image dehazing and beyond [J]. IEEE Transactions on Image Processing , 2019 , 28 ( 1 ): 492 - 505 .

WANG K , ZHUO L , LI J F , et al . Learning an enhancement convolutional neural network for multi-degraded images [J]. Sensing and Imaging , 2020 , 21 ( 1 ): 1 - 15 .

ZHAO S Y , ZHANG L , HUANG S Y , et al . Dehazing evaluation: Real-world benchmark datasets, criteria, and baselines [J]. IEEE Transactions on Image Processing , 2020 , 29 : 6947 - 6962 .

SILBERMAN N , HOIEM D , KOHLI P , et al . Indoor segmentation and support inference from RGBD images [C]// European Conference on Computer Vision . Florence : Springer , 2012 : 746 - 760 .

韩翰 , 卓力 , 张菁 , 等 . 基于深度学习的无参考图像质量评价综述 [J]. 测控技术 , 2022 , 41 ( 4 ): 1 - 10 .

HAN H , ZHUO L , ZHANG J , et al . Summary of no-reference image quality assessment based on deep learning [J]. Measurement & Control Technology , 2022 , 41 ( 4 ): 1 - 10 . (in Chinese)

MITTAL A , MOORTHY A K , BOVIK A C . No-reference image quality assessment in the spatial domain [J]. IEEE Transactions on Image Processing , 2012 , 21 ( 12 ): 4695 - 4708 .

MITTAL A , SOUNDARARAJAN R , BOVIK A C . Making a “completely blind” image quality analyzer [J]. IEEE Signal Processing Letters , 2013 , 20 ( 3 ): 209 - 212 .

ZHU H C , LI L D , WU J J , et al . MetaIQA: Deep meta-learning for no-reference image quality assessment [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 14131 - 14140 .

MIN X K , ZHAI G T , GU K , et al . Objective quality evaluation of dehazed images [J]. IEEE Transactions on Intelligent Transportation Systems , 2019 , 20 ( 8 ): 2879 - 2892 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于尺度线索增强的无监督单目深度估计

一种基于修正动量的RBM算法

面向时序异常检测的可变视距多向扫描方法

基于稀疏平滑自蒸馏的差分隐私深度学习方法