基于分频式生成对抗网络的非成对水下图像增强

牛玉贞; 张凌昕; 兰杰; 许瑞; 柯逍

doi:10.12263/DZXB.20240265

您当前的位置：

首页 >

文章列表页 >

基于分频式生成对抗网络的非成对水下图像增强

学术论文 | 更新时间：2025-12-08

- 基于分频式生成对抗网络的非成对水下图像增强
- FD-GAN: Frequency-Decomposed Generative Adversarial Network for Unpaired Underwater Image Enhancement
- 电子学报 2025年53卷第2期页码：527-544
- 作者机构：
  
  福州大学计算机与大数据学院，福建福州 350108
- 作者简介：
  
  [ "牛玉贞女，1982年出生于山东省济南市，博士.现为福州大学计算机与大数据学院教授、博士生导师.主要研究方向为计算机视觉. E-mail: yuzhenniu@gmail.com" ]
  [ "张凌昕女，1999年出生于福建省福州市.现为福州大学计算机与大数据学院硕士研究生.主要研究方向为计算机视觉. E-mail: fzuzhanglingxin@163.com" ]
  [ "兰杰男，2000年出生于福建省武平县.现为福州大学计算机与大数据学院硕士研究生.主要研究方向为计算机视觉. E-mail: jielan.blue@gmail.com" ]
  [ "许瑞男，1995年出生于江苏省扬州市.现为福州大学计算机与大数据学院博士研究生.主要研究方向为图像增强与复原、视觉目标跟踪. E-mail: xurui.ryan.chn@gmail.com" ]
  [ "柯逍男，1982年出生福建省福州市，博士.现为福州大学教授、博士生导师.主要研究方向为计算机视觉、模式识别. E-mail: kex@fzu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(U21A20472;61972097);国家重点研发计划(2021YFB3600503);福建省科技重大专项(2021HZ022007);福建省自然科学基金(2023J01067;2020J01494);福建省科技厅高校产学合作项目(2021H6022)
- DOI：10.12263/DZXB.20240265
  中图分类号： TP391;
- 收稿：2024-03-25，
  
  修回：2024-11-09，
  
  纸质出版：2025-02-25
- 稿件说明：
移动端阅览
牛玉贞, 张凌昕, 兰杰, 等. 基于分频式生成对抗网络的非成对水下图像增强[J]. 电子学报, 2025, 53(02): 527-544.

NIU Yu-zhen, ZHANG Ling-xin, LAN Jie, et al. FD-GAN: Frequency-Decomposed Generative Adversarial Network for Unpaired Underwater Image Enhancement[J]. Acta Electronica Sinica, 2025, 53(02): 527-544.
牛玉贞, 张凌昕, 兰杰, 等. 基于分频式生成对抗网络的非成对水下图像增强[J]. 电子学报, 2025, 53(02): 527-544. DOI：10.12263/DZXB.20240265

NIU Yu-zhen, ZHANG Ling-xin, LAN Jie, et al. FD-GAN: Frequency-Decomposed Generative Adversarial Network for Unpaired Underwater Image Enhancement[J]. Acta Electronica Sinica, 2025, 53(02): 527-544. DOI：10.12263/DZXB.20240265

摘要

增强水下图像质量对水下作业领域的发展具有重要意义.现有的水下图像增强方法通常基于成对的水下图像和参考图像进行训练，然而实际获取与水下图像对应的参考图像比较困难，相比之下获得非成对高质量水下图像或者陆上图像较为容易.此外，现有的水下图像增强方法很难同时针对各种失真类型进行图像增强.为了避免对成对训练数据的依赖和进一步降低获得训练数据的难度，并应对多样的水下图像失真类型，本文提出了一种基于分频式生成对抗网络（Frequency-Decomposed Generative Adversarial Network，FD-GAN）的非成对水下图像增强方法，并在此基础上设计了高低频双分支生成器用于重建高质量水下增强图像.具体来说，本文引入特征级别的小波变换将特征分为低频和高频部分，并基于循环一致性生成对抗网络对低频和高频部分区分处理.其中，低频分支采用结合低频注意力机制的编码-解码器结构实现对图像颜色和亮度的增强，高频分支则采用并行的高频注意力机制对各高频分量进行增强，从而实现对图像细节的恢复.在多个标准水下图像数据集上的实验结果表明，本文提出的方法在使用非成对的高质量水下图像和引入部分陆上图像的情况下，均能有效生成高质量的水下增强图像，且有效性和泛化性均优于当前主流的水下图像增强方法.

Abstract

Enhancing the quality of underwater images is crucial for advancements in the fields of underwater exploration and underwater rescue. Existing underwater image enhancement methods typically rely on paired underwater images and reference images for training. However

obtaining corresponding reference images for underwater images is challenging in practice. In contrast

acquiring high-quality unpaired underwater images or images captured on land are relatively more straightforward. Furthermore

existing techniques for underwater image enhancement often struggle to address a variety of distortion types simultaneously. To avoid the reliance on paired training data

reduce the difficulty of acquiring training data

and effectively handle diverse types of underwater image distortions

in this paper

we propose a novel unpaired underwater image enhancement method based on the frequency-decomposed generative adversarial network (FD-GAN). We design a dual-branch generator based on high and low frequencies to reconstruct high-quality underwater images. Specifically

feature-level wavelet transform is introduced to separate the features into low-frequency and high-frequency parts. Then the separated features are processed by a cycle-consistent generative adversarial network

so as to simultaneously enhance the color and luminance in the low-frequency component and details in the high-frequency part. More specific

the low-frequency branch employs an encoder-decoder structure with a low-frequency attention mechanism to enhance the color and brightness of the image. The high-frequency branch utilizes parallel high-frequency attention mechanisms to enhance various high-frequency components

thereby achieving the restoration of image details. Experimental results on multiple datasets show that the proposed method trained with unpaired high-quality underwater images or unpaired high-quality underwater images and on-land images

can effectively generate high-quality underwater enhanced images and the proposed method is superior to the state-of-the-art underwater image enhancement methods in terms of effectiveness and generalization.

关键词

Keywords

references

IQBAL K , ODETAYO M , JAMES A , et al . Enhancing the low quality images using unsupervised colour correction method [C ] // 2010 IEEE International Conference on Systems, Man and Cybernetics . Piscataway : IEEE , 2010 : 1703 - 1709 .

刘柯 , 李旭健 . 水下和微光图像的去雾及增强方法 [J ] . 光学学报 , 2020 , 40 ( 19 ): 73 - 85 .

LIU K , LI X J . De-hazing and enhancement methods for underwater and low-light images [J ] . Acta Optica Sinica , 2020 , 40 ( 19 ): 73 - 85 . (in Chinese)

牛玉贞 , 陈铭铭 , 李悦洲 , 等 . 基于任务解耦的低照度图像增强方法 [J ] . 电子学报 , 2024 , 52 ( 1 ): 34 - 45 .

NIU Y Z , CHEN M M , LI Y Z , et al . Task decoupling guided low-light image enhancement [J ] . Acta Electronica Sinica , 2024 , 52 ( 1 ): 34 - 45 . (in Chinese)

肖进胜 , 申梦瑶 , 雷俊锋 , 等 . 基于生成对抗网络的雾霾场景图像转换算法 [J ] . 计算机学报 , 2020 , 43 ( 1 ): 165 - 176 .

XIAO J S , SHEN M Y , LEI J F , et al . Image conversion algorithm for haze scene based on generative adversarial networks [J ] . Chinese Journal of Computers , 2020 , 43 ( 1 ): 165 - 176 . (in Chinese)

徐少平 , 陈孝国 , 李芬 , 等 . 采用两阶段混合策略实现的低照度图像增强算法 [J ] . 电子学报 , 2021 , 49 ( 11 ): 2166 - 2170 .

XU S P , CHEN X G , LI F , et al . A low-light image enhancement algorithm using two-stage hybrid strategy [J ] . Acta Electronica Sinica , 2021 , 49 ( 11 ): 2166 - 2170 . (in Chinese)

LI J , SKINNER K A , EUSTICE R M , et al . WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 1 ): 387 - 394 .

FABBRI C , ISLAM M J , SATTAR J . Enhancing underwater imagery using generative adversarial networks [C ] // 2018 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway : IEEE , 2018 : 7159 - 7165 .

LI C Y , GUO C L , REN W Q , et al . An underwater image enhancement benchmark dataset and beyond [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 4376 - 4389 .

ISLAM M J , XIA Y Y , SATTAR J . Fast underwater image enhancement for improved visual perception [J ] . IEEE Robotics and Automation Letters , 2020 , 5 ( 2 ): 3227 - 3234 .

NAIK A , SWARNAKAR A , MITTAL K . Shallow-UWnet: Compressed model for underwater image enhancement (student abstract) [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 18 ): 15853 - 15854 .

ISLAM M J , LUO P G , SATTAR J . Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception [EB/OL ] . ( 2020-02-04 )[ 2024-03-24 ] . https://arxiv.org/abs/2002.01155v1 https://arxiv.org/abs/2002.01155v1 .

LI C Y , ANWAR S , PORIKLI F . Underwater scene prior inspired deep underwater image and video enhancement [J ] . Pattern Recognition , 2020 , 98 : 107038 .

WANG Z Y , SHEN L Q , XU M , et al . Domain adaptation for underwater image enhancement [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 1442 - 1457 .

LIU S B , FAN H J , LIN S , et al . Adaptive learning attention network for underwater image enhancement [J ] . IEEE Robotics and Automation Letters , 2022 , 7 ( 2 ): 5326 - 5333 .

FU Z Q , WANG W , HUANG Y , et al . Uncertainty inspired underwater image enhancement [M ] // Lecture Notes in Computer Science . Cham : Springer Nature Switzerland , 2022 : 465 - 482 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 2242 - 2251 .

LI B , CHEN X F . Wavelet-based numerical analysis: A review and classification [J ] . Finite Elements in Analysis and Design , 2014 , 81 : 14 - 31 .

LIU R S , FAN X , ZHU M , et al . Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 30 ( 12 ): 4861 - 4875 .

BYCHKOVSKY V , PARIS S , CHAN E , et al . Learning photographic global tonal adjustment with a database of input/output image pairs [C ] // 2011 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2011 : 97 - 104 .

ABDUL GHANI A S , MAT ISA N A . Underwater image quality enhancement through integrated color model with rayleigh distribution [J ] . Applied Soft Computing , 2015 , 27 : 219 - 230 .

ANCUTI C O , ANCUTI C , DE VLEESCHOUWER C , et al . Color balance and fusion for underwater image enhancement [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 1 ): 379 - 393 .

ANCUTI C O , ANCUTI C , DE VLEESCHOUWER C , et al . Color channel compensation (3C): A fundamental pre-processing step for image enhancement [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 2653 - 2665 .

FU X Y , ZHUANG P X , HUANG Y , et al . A retinex-based enhancing approach for single underwater image [C ] // 2014 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2014 : 4572 - 4576 .

ZHANG S , WANG T , DONG J Y , et al . Underwater image enhancement via extended multi-scale Retinex [J ] . Neurocomputing , 2017 , 245 : 1 - 9 .

XIE B , GUO F , CAI Z X . Improved single image dehazing using dark channel prior and multi-scale retinex [C ] // 2010 International Conference on Intelligent System Design and Engineering Application . Piscataway : IEEE , 2010 : 848 - 851 .

CHIANG J Y , CHEN Y C . Underwater image enhancement by wavelength compensation and dehazing [J ] . IEEE Transactions on Image Processing , 2012 , 21 ( 4 ): 1756 - 1769 .

DREWS P L J JR , NASCIMENTO E R , BOTELHO S S C , et al . Underwater depth estimation and image restoration based on single images [J ] . IEEE Computer Graphics and Applications , 2016 , 36 ( 2 ): 24 - 35 .

PENG Y T , CAO K M , COSMAN P C . Generalization of the dark channel prior for single image restoration [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 6 ): 2856 - 2868 .

BERMAN D , LEVY D , AVIDAN S , et al . Underwater single image color restoration using haze-lines and a new quantitative dataset [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 8 ): 2822 - 2837 .

JIANG Y F , ZHANG H , ZHANG J M , et al . SSH: A self-supervised framework for image harmonization [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4812 - 4821 .

陈勇 , 陈东 , 刘焕淋 , 等 . 基于深度卷积神经网络的无参考低照度图像增强 [J ] . 电子与信息学报 , 2022 , 44 ( 6 ): 2166 - 2174 .

CHEN Y , CHEN D , LIU H L , et al . Unreferenced low-lighting image enhancement based on deep convolutional neural network [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 6 ): 2166 - 2174 . (in Chinese)

DAN R T , MOTIIAN S , FAIETA B , et al . ALADIN: All layer adaptive instance normalization for fine-grained style similarity [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 11906 - 11915 .

NESHATAVAR R , YAVARTANOO M , SON S , et al . CVF-SID: Cyclic multi-variate function for self-supervised image denoising by disentangling noise from image [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 17562 - 17570 .

ANCUTI C , ANCUTI C O , HABER T , et al . Enhancing underwater images and videos by fusion [C ] // 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2012 : 81 - 88 .

GUO Y C , LI H Y , ZHUANG P X . Underwater image enhancement using a multiscale dense generative adversarial network [J ] . IEEE Journal of Oceanic Engineering , 2020 , 45 ( 3 ): 862 - 870 .

RONNEBERGER O , FISCHER P , BROX T . U-Net: Convolutional networks for biomedical image segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2015 : 234 - 241 .

JIANG N F , CHEN W L , LIN Y T , et al . Underwater image enhancement with lightweight cascaded network [J ] . IEEE Transactions on Multimedia , 2022 , 24 : 4301 - 4313 .

ZHUANG P X , WU J M , PORIKLI F , et al . Underwater image enhancement with hyper-Laplacian reflectance priors [J ] . IEEE Transactions on Image Processing , 2022 , 31 : 5442 - 5455 .

ZHOU J C , LI B S , ZHANG D H , et al . UGIF-net: An efficient fully guided information flow network for underwater image enhancement [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2023 , 61 : 4206117 .

RHIF M , ABBES A BEN , FARAH I R , et al . Wavelet transform application for/in non-stationary time-series analysis: A review [J ] . Applied Sciences , 2019 , 9 ( 7 ): 1345 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

JADERBERG M , SIMONYAN K , ZISSERMAN A . Spatial transformer networks [C ] // Proceedings of the Advances in Neural Information Processing Systems . Quebec : NIPS Foundation , 2015 : 8024 - 8035 .

PASZKE A , GROSS S , MASSA F , et al . Pytorch: An imperative style, high-performance deep learning library [C ] // Proceedings of the Advances in Neural Information Processing Systems . British Columbia : NIPS Foundation , 2019 : 2017 - 2025 .

KINGMA D P , BA J , HAMMAD M M . Adam: A method for stochastic optimization [EB/OL ] . ( 2021-12-22 )[ 2024-03-24 ] . https://arxiv.org/abs/1412.6980v9 https://arxiv.org/abs/1412.6980v9 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2018 : 3 - 19 .

KIRILLOV A , MINTUN E , RAVI N , et al . Segment anything [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 3992 - 4003 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于改进Res2Net与自适应多尺度窗口池化的调制识别方法

基于重复感知与频率增强的动态图推荐研究

基于序列与跨模态对齐的蛋白质功能预测模型

NAHGNN：邻域感知异构图神经网络