Progressive Edge-Aware Interactive Network for Multi-Degraded Low-Light Image Enhancement

LI Yue-zhou; NIU Yu-zhen; LI Fu-sheng; KE Xiao; SHI Yi-qing

doi:10.12263/DZXB.20240375

您当前的位置：

首页 >

文章列表页 >

Progressive Edge-Aware Interactive Network for Multi-Degraded Low-Light Image Enhancement

PAPERS | 更新时间：2025-12-08

- Progressive Edge-Aware Interactive Network for Multi-Degraded Low-Light Image Enhancement
- ACTA ELECTRONICA SINICA Vol. 53, Issue 3, Pages: 926-940(2025)
- 作者机构：
  
  1.福州大学计算机与大数据学院，福建福州 350108
  2.福建师范大学光电与信息工程学院，福建福州 350117
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(U21A20472;62072110;61972097);Natural Science Foundation of Fujian Province(2021HZ022007);Natural Science Foundation of Fujian Province(2023J01067;2020J01494);Industry-Academy Cooperation Project of Fujian Province (China)(2021H6022)
- DOI：10.12263/DZXB.20240375
  CLC： TP391;
- Received：23 April 2024，
  
  Revised：2025-02-21，
  
  Published：25 March 2025
- 稿件说明：
移动端阅览
李悦洲, 牛玉贞, 李富晟, 等. 基于渐进式边缘感知交互的多退化暗光图像增强[J]. 电子学报, 2025, 53(03): 926-940.

LI Yue-zhou, NIU Yu-zhen, LI Fu-sheng, et al. Progressive Edge-Aware Interactive Network for Multi-Degraded Low-Light Image Enhancement[J]. Acta Electronica Sinica, 2025, 53(03): 926-940.
李悦洲, 牛玉贞, 李富晟, 等. 基于渐进式边缘感知交互的多退化暗光图像增强[J]. 电子学报, 2025, 53(03): 926-940. DOI：10.12263/DZXB.20240375

LI Yue-zhou, NIU Yu-zhen, LI Fu-sheng, et al. Progressive Edge-Aware Interactive Network for Multi-Degraded Low-Light Image Enhancement[J]. Acta Electronica Sinica, 2025, 53(03): 926-940. DOI：10.12263/DZXB.20240375

摘要

暗光场景下拍摄的图像容易受暗光、噪声、模糊等多种退化因素的影响，导致其内容可视度低且视觉观感较差.多退化的暗光图像对现有图像增强方法提出了挑战：一方面，暗光图像增强或去模糊等方法不能涵盖所有的退化类型，而组合使用已有方法的效果受到计算开销增加与误差累积的限制；另一方面，已有多退化暗光图像增强方法采用了先提升亮度再去除模糊的策略，这种顺序处理方式会增加特征线索丢失的风险，不利于细节复原.为应对上述挑战，本文提出渐进式边缘感知交互增强网络（Progressive Edge-aware Interactive Enhancement Network，PEIE-Net），以逐步增强的方式减少图像增强过程中特征细节的丢失.具体来说，该网络由图像增强分支与边缘预测分支组成.在图像增强分支的每个增强阶段中，设计自注意力调制预测模块提取全局信息，用于对通道调制模块和多尺度复原模块进行自适应调制.在边缘预测分支中，设计空频域特征变换模块提取边缘感知信息，既用于预测高质量图像的边缘，又与图像增强分支的特征进行融合，以此模拟人类视觉系统在不同感知之间的交互.此外，本文还提出了场景亮度估计损失对多个渐进式增强阶段进行协调.在合成与真实数据集上的实验验证了本文方法在增强暗光、噪声、模糊退化图像方面的有效性与先进性，并可用于暗光图像增强与超分辨率任务.

Abstract

Images captured in low-light scenes are susceptible to multiple degradations such as darkness

noise

and blur

resulting in poor visibility and visual perception. Multi-degraded low-light image enhancement poses challenges to existing image enhancement methods as follows: on the one hand

low-light image enhancement or deblurring methods cannot handle all three types of degradation

and the effect of the combination strategy is limited by the increased computational cost and error accumulation. On the other hand

the existing multi-degraded low-light image enhancement method adopts the strategy of enhancing brightness first and then removing blur

and this sequential processing manner increases the risk of losing feature cues and is not conducive to detail recovery. To cope with the above challenges

this paper proposes the progressive edge-aware interactive enhancement network (PEIE-Net)

which reduces the loss of feature details by designing a step-by-step enhancement process. Specifically

our network consists of an image enhancement branch and an edge information prediction branch. In each enhancement stage of the image enhancement branch

a self-attention modulation prediction module is designed to extract the global information

which is used for adaptive modulation in the channel modulation module and multi-scale restoration module. In the edge information prediction branch

the spatial-frequency domain feature transformation module is developed to extract the edge perceptual information. The edge perceptual information is used to predict the edges of high-quality images while also fused with the image enhancement features

simulating the interaction between different perceptions within the human visual system. In addition

we propose scene brightness estimation loss to coordinate the multiple progressive enhancement stages. Experiments on synthetic and real datasets demonstrate the effectiveness and sophistication of our method for enhancing low-light

noisy

and blur-degraded images

and can be used for low-light image enhancement and super-resolution tasks.

关键词

Keywords

references

江泽涛 , 施道权 , 雷晓春 , 等 . 一种基于Night-YOLOX的低照度目标检测方法 [J ] . 电子学报 , 2023 , 51 ( 10 ): 2821 - 2830 .

JIANG Z T , SHI D Q , LEI X C , et al . A low-illumination object detection method based on Night-YOLOX [J ] . Acta Electronica Sinica , 2023 , 51 ( 10 ): 2821 - 2830 . (in Chinese)

LI Y Z , NIU Y Z , XU R , et al . Zero-referenced enlightening and restoration for UAV nighttime vision [J ] . IEEE Geoscience and Remote Sensing Letters , 2024 , 21 : 8002105 .

牛玉贞 , 林晓锋 , 许煌标 , 等 . 基于Transformer的多尺度优化低照度图像增强网络 [J ] . 模式识别与人工智能 , 2023 , 36 ( 6 ): 511 - 529 .

NIU Y Z , LIN X F , XU H B , et al . Transformer-based multi-scale optimization network for low-light image enhancement [J ] . Pattern Recognition and Artificial Intelligence , 2023 , 36 ( 6 ): 511 - 529 . (in Chinese)

WANG W J , XU Z B , HUANG H F , et al . Self-aligned concave curve: Illumination enhancement for unsupervised adaptation [C ] // Proceedings of the 30th ACM International Conference on Multimedia . New York : ACM , 2022 : 2617 - 2626 .

WANG Y F , WAN R J , YANG W H , et al . Low-light image enhancement with normalizing flow [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 3 ): 2604 - 2612 .

LIU R S , MA L , MA T Y , et al . Learning with nested scene modeling and cooperative architecture search for low-light vision [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 5 ): 5953 - 5969 .

江泽涛 , 钱艺 , 伍旭 , 等 . 一种基于ARD-GAN的低照度图像增强方法 [J ] . 电子学报 , 2021 , 49 ( 11 ): 2160 - 2165 .

JIANG Z T , QIAN Y , WU X , et al . Low-light image enhancement method based on ARD-GAN [J ] . Acta Electronica Sinica , 2021 , 49 ( 11 ): 2160 - 2165 . (in Chinese)

ZHOU S C , LI C Y , CHANGE LOY C . LEDNet: Joint low-light enhancement and deblurring in the dark [M ] // Computer Vision-ECCV 2022 . Cham : Springer Nature Switzerland , 2022 : 573 - 589 .

WANG T , ZHANG K H , SHEN T R , et al . Ultra-high-definition low-light image enhancement: A benchmark and transformer-based method [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 3 ): 2654 - 2662 .

CHO S J , JI S W , HONG J P , et al . Rethinking coarse-to-fine approach in single image deblurring [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4621 - 4630 .

YE J , LIU Y , YU C J , et al . ASP-LED: Learning ambiguity-aware structural priors for joint low-light enhancement and deblurring [C ] // 2024 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway : IEEE , 2024 : 12389 - 12396 .

ZOU W B , GAO H X , YE T , et al . VQCNIR: Clearer night image restoration with vector-quantized codebook [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2024 , 38 ( 7 ): 7873 - 7881 .

AAKERBERG A , NASROLLAHI K , MOESLUND T B . RELLISUR: A real low-light image super-resolution dataset [C ] // In Advances in Neural Information Processing Systems 35 (NeurIPS 2021) . Schloss Dgstuhl : DBLP , 2021 : 5234969 .

WEI C , WANG W , YANG W , et al . Deep retinex decomposition for low-light enhancement [C ] // Proceedings of the Conference on British Machine Vision Conference . Newcastle : BMVC , 2018 : 1 - 12 .

ZHANG Y H , ZHANG J W , GUO X J . Kindling the darkness: A practical low-light image enhancer [C ] // Proceedings of the 27th ACM International Conference on Multimedia . New York : ACM , 2019 : 1632 - 1640 .

CAI Y H , BIAN H , LIN J , et al . Retinexformer: One-stage retinex-based transformer for low-light image enhancement [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 12470 - 12479 .

LIM S , KIM W . DSLR: Deep stacked Laplacian restorer for low-light image enhancement [J ] . IEEE Transactions on Multimedia , 2020 , 23 : 4272 - 4284 .

XU X G , WANG R X , FU C W , et al . SNR-aware low-light image enhancement [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 17693 - 17703 .

LIU Y B , HARIDEVAN A , SCHOFIELD H , et al . Application of ghost-DeblurGAN to fiducial marker detection [C ] // 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE , 2022 : 6827 - 6832 .

YAN Q S , GONG D , WANG P , et al . SharpFormer: Learning local feature preserving global representations for image deblurring [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 2857 - 2866 .

白勇强 , 禹晶 , 李一秾 , 等 . 基于深度先验的盲图像去模糊算法 [J ] . 电子学报 , 2023 , 51 ( 4 ): 1050 - 1067 .

BAI Y Q , YU J , LI Y N , et al . Deep prior-based blind image deblurring [J ] . Acta Electronica Sinica , 2023 , 51 ( 4 ): 1050 - 1067 . (in Chinese)

XING W Z , EGIAZARIAN K . End-to-end learning for joint image demosaicing, denoising and super-resolution [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 3506 - 3515 .

NIU W J , ZHANG K H , LUO W H , et al . Blind motion deblurring super-resolution: When dynamic spatio-temporal learning meets static image understanding [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 7101 - 7111 .

YANG X , WANG X Q , WANG N N , et al . SRDN: A unified super-resolution and motion deblurring network for space image restoration [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2021 , 60 : 5614411 .

WANG R , ZHANG C J , ZHENG X L , et al . Joint defocus deblurring and superresolution learning network for autonomous driving [J ] . IEEE Intelligent Transportation Systems Magazine , 2024 , 16 ( 1 ): 104 - 115 .

RASHEED M T , SHI D M . LSR: Lightening super-resolution deep network for low-light image enhancement [J ] . Neurocomputing , 2022 , 505 : 263 - 275 .

ZHANG Y , TSANG I W , LUO Y W , et al . Recursive copy and paste GAN: Face hallucination from shaded thumbnails [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 8 ): 4321 - 4338 .

YE J , YANG L J , QIU C Z , et al . Joint low-light enhancement and deblurring with structural priors guidance [J ] . Expert Systems with Applications , 2024 , 249 : 123722 .

LV X Q , ZHANG S P , WANG C Y , et al . Fourier priors-guided diffusion for zero-shot joint low-light enhancement and deblurring [C ] // 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2024 : 25378 - 25388 .

牛玉贞 , 陈铭铭 , 李悦洲 , 等 . 基于任务解耦的低照度图像增强方法 [J ] . 电子学报 , 2024 , 52 ( 1 ): 34 - 45 .

NIU Y Z , CHEN M M , LI Y Z , et al . Task decoupling guided low-light image enhancement [J ] . Acta Electronica Sinica , 2024 , 52 ( 1 ): 34 - 45 . (in Chinese)

YI R , TIAN H Y , GU Z H , et al . Towards artistic image aesthetics assessment: A large-scale dataset and a new method [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 22388 - 22397 .

WANG L G , WANG Y Q , DONG X Y , et al . Unsupervised degradation representation learning for blind super-resolution [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE , 2021 : 10581 - 10590 .

BERTINETTO L , HENRIQUES J F , VALMADRE J , et al . Learning feed-forward one-shot learners [EB/OL ] . ( 2016-01-16 )[ 2024-04-07 ] . https://arxiv.org/abs/1606.05233v1 https://arxiv.org/abs/1606.05233v1 .

LI X J , HUANG L , WEI G Q , et al . Online parallel framework for real-time visual tracking [J ] . Engineering Applications of Artificial Intelligence , 2021 , 102 : 104266 .

CUI Z T , LI K C , GU L , et al . You only need 90K parameters to adapt light: A light weight transformer for image enhancement and exposure correction [EB/OL ] . ( 2022-10-08 )[ 2024-04-07 ] . https://arxiv.org/abs/2205.14871v4 https://arxiv.org/abs/2205.14871v4 .

LI B , WU W , WANG Q , et al . SiamRPN++: Evolution of Siamese visual tracking with very deep networks [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 4282 - 4291 .

MILDENHALL B , BARRON J T , CHEN J W , et al . Burst denoising with kernel prediction networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 2502 - 2510 .

WEI X S , LUO J H , WU J X , et al . Selective convolutional descriptor aggregation for fine-grained image retrieval [J ] . IEEE Transactions on Image Processing , 2017 , 26 ( 6 ): 2868 - 2881 .

WANG H H , WU X D , HUANG Z Y , et al . High-frequency component helps explain the generalization of convolutional neural networks [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 8684 - 8694 .

GONZALEZ R C , WINTZ P A . Digital Image Processing [M ] . Mass .: Addison-Wesley Pub. Co. , 1977 .

WANG C X , WU H J , JIN Z . FourLLIE: Boosting low-light image enhancement by Fourier frequency information [C ] // Proceedings of the 31st ACM International Conference on Multimedia . New York : ACM , 2023 : 7459 - 7469 .

KONG L S , DONG J X , GE J J , et al . Efficient frequency domain-based transformers for high-quality image deblurring [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 5886 - 5895 .

ZAMIR S W , ARORA A , KHAN S , et al . Multi-stage progressive image restoration [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 14816 - 14826 .

ZHANG Y , YANG Q X , CHANDLER D M , et al . Reference-based multi-stage progressive restoration for multi-degraded images [J ] . IEEE Transactions on Image Processing , 2024 , 33 : 4982 - 4997 .

WAN Y C , SHAO M W , CHENG Y S , et al . Progressive convolutional transformer for image restoration [J ] . Engineering Applications of Artificial Intelligence , 2023 , 125 : 106755 .

ZHANG Y L , LI K P , LI K , et al . Image super-resolution using very deep residual channel attention networks [M ] // Computer Vision-ECCV 2018 . Cham : Springer International Publishing , 2018 : 294 - 310 .

CHENG D Q , CHEN L L , LV C , et al . Light-guided and cross-fusion U-Net for anti-illumination image super-resolution [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 12 ): 8436 - 8449 .

GAO J X , YUE Z Y , LIU Y H , et al . Diving into darkness: A dual-modulated framework for high-fidelity super-resolution in ultra-dark environments [EB/OL ] . ( 2023-09-11 )[ 2024-04-07 ] . https://arxiv.org/abs/2309.05267v1 https://arxiv.org/abs/2309.05267v1 .

GAO J , LIU Y , YUE Z , FAN X , et al . Collaborative brightening and amplification of low-light imagery via bi-level adversarial learning [EB/OL ] . ( 2023-10-03 )[ 2024-04-07 ] . https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4617171 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4617171 .

YUE Z Y , GAO J X , XIE S H , et al . LoLiSRFlow: Joint single image low-light enhancement and super-resolution via cross-scale transformer-based conditional flow [EB/OL ] . ( 2024-02-29 )[ 2024-04-07 ] . https://arxiv.org/abs/2402.18871v1 https://arxiv.org/abs/2402.18871v1 .

CHEN X Y , WANG X T , ZHOU J T , et al . Activating more pixels in image super-resolution transformer [C ] // Proceedings of the Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 22367 - 22377 .

ZHENG M J , SUN L , DONG J X , et al . SMFANet: A lightweight self-modulation feature aggregation network for efficient image super-resolution [M ] // Computer Vision-ECCV 2024 . Cham : Springer Nature Switzerland , 2024 : 359 - 375 .

ZHOU Y P , LI Z , GUO C L , et al . SRFormer: Permuted self-attention for single image super-resolution [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 12734 - 12745 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer: Efficient transformer for high-resolution image restoration [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 5718 - 5729 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image Enhancement via Content Semantic-Aware Multimodal Fusion

Quality Assessment of Light Field Images Based on Contrastive Visual-Textual Model

Infrared Image Enhancement Based on Multi-Scale Cyclic Convolution and Multi-Clustering Space

Dynamic Multi-Histogram Equalization Based on Fast Fuzzy Clustering

Related Author

LI Yue-zhou

NIU Yu-zhen

LI Fu-sheng

KE Xiao

SHI Yi-qing

ZHU Han-cheng

LIU Xin-yu

YAO Rui

Related Institution

School of Artificial Intelligence, Xidian University

Mine Digitization Engineering Research Center of the Ministry of Education

School of Computer Science and Technology / School of Artificial Intelligence, China University of Mining and Technology

Engineering Research Center of Big Data Intelligence, Ministry of Education

⁰