Task Decoupling Guided Low-Light Image Enhancement

NIU Yu-zhen; CHEN Ming-ming; LI Yue-zhou; ZHAO Tie-song

doi:10.12263/DZXB.20230189

您当前的位置：

首页 >

文章列表页 >

Task Decoupling Guided Low-Light Image Enhancement

PAPERS | 更新时间：2026-04-10

- Task Decoupling Guided Low-Light Image Enhancement
- ACTA ELECTRONICA SINICA Vol. 52, Issue 1, Pages: 34-45(2024)
- 作者机构：
  
  1.福州大学计算机与大数据学院，福建福州 350108
  2.福建省网络计算与智能信息处理重点实验室（福州大学），福建福州 350108
  3.大数据智能教育部工程研究中心，福建福州 350108
  4.福州大学物理与信息工程学院，福建福州 350108
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62171134);Natural Science Foundation of Fujian Province, China(2023J01067)
- DOI：10.12263/DZXB.20230189
  CLC： TP391.41
- Received：03 March 2023，
  
  Revised：2023-12-14，
  
  Published：25 January 2024
- 稿件说明：
移动端阅览
牛玉贞,陈铭铭,李悦洲,等.基于任务解耦的低照度图像增强方法[J].电子学报,2024,52(01):34-45.

NIU Yu-zhen, CHEN Ming-ming, LI Yue-zhou, et al.Task Decoupling Guided Low-Light Image Enhancement[J].Acta Electronica Sinica, 2024, 52(01): 34-45.
牛玉贞,陈铭铭,李悦洲,等.基于任务解耦的低照度图像增强方法[J].电子学报,2024,52(01):34-45. DOI：10.12263/DZXB.20230189

NIU Yu-zhen, CHEN Ming-ming, LI Yue-zhou, et al.Task Decoupling Guided Low-Light Image Enhancement[J].Acta Electronica Sinica, 2024, 52(01): 34-45. DOI：10.12263/DZXB.20230189

摘要

低照度条件下拍摄的照片往往存在亮度低、颜色失真、噪声高、细节退化等多重耦合问题，因此低照度图像增强是一个具有挑战性的任务.现有基于深度学习的低照度图像增强方法通常聚焦于对亮度和色彩的提升，导致增强图像中仍然存在噪声等缺陷.针对上述问题，本文提出了一种基于任务解耦的低照度图像增强方法，根据低照度图像增强任务对高层和低层特征的不同需求，将该任务解耦为亮度与色彩增强和细节重构两组任务，进而构建双分支低照度图像增强网络模型（Two-Branch Low-light Image Enhancement Network，TBLIEN）.其中，亮度与色彩增强分支采用带全局特征的U-Net结构，提取深层语义信息改善亮度与色彩；细节重构分支采用保持原始分辨率的全卷积网络实现细节复原和噪声去除.此外，在细节重构分支中，本文提出一种半双重注意力残差模块，能在保留上下文特征的同时通过空间和通道注意力强化特征，从而实现更精细的细节重构.在合成和真实数据集上的广泛实验表明，本文模型的性能超越了当前先进的低照度图像增强方法，并具有更好的泛化能力，且可适用于水下图像增强等其他图像增强任务.

Abstract

Photos captured under low-light conditions suffer from multiple coupling problems

i.e.

low brightness

color distortion

heavy noise

and detail degradation

making low-light image enhancement a challenging task. Existing deep learning-based low-light image enhancement methods typically focus on improving the illumination and color while neglecting the noise in the enhanced image. To solve the above problems

this paper proposes a low-light image enhancement method based on task decoupling. According to the different requirements for high-level and low-level features

the low-light image enhancement task is decoupled into two subtasks: illumination and color enhancement and detail reconstruction. Based on the task decoupling

we propose a two-branch low-light image enhancement network (TBLIEN). The illumination and color enhancement branch is built as a U-Net structure with global feature extraction

which exploits deep semantic information for illumination and color improvement. The detail reconstruction branch uses a fully convolutional network that preserves the original resolution while performing detail restoration and noise removal. In addition

for the detail reconstruction branch

we design a half-dual attention residual module. Our module enhances features through spatial and channel attention mechanisms while preserving their context

allowing precise detail reconstruction. Extensive experiments on real and synthetic datasets show that our model outperforms other state-of-the-art methods

and has better generalization capability. Our method is also applicable to other image enhancement tasks

i.e.

underwater image enhancement.

关键词

Keywords

references

SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 4 ): 640 - 651 .

陈科圻 , 朱志亮 , 邓小明 , 等 . 多尺度目标检测的深度学习研究综述 [J]. 软件学报 , 2021 , 32 ( 4 ): 1201 - 1227 .

CHEN K Q , ZHU Z L , DENG X M , et al . Deep learning for multi-scale object detection: A survey [J]. Journal of Software , 2021 , 32 ( 4 ): 1201 - 1227 . (in Chinese)

LIM J , HEO M , LEE C , et al . Contrast enhancement of noisy low-light images based on structure-texture-noise decomposition [J]. Journal of Visual Communication and Image Representation , 2017 , 45 : 107 - 121 .

CHENG H D , SHI X J . A simple and effective histogram equalization approach to image enhancement [J]. Digital Signal Processing , 2004 , 14 ( 2 ): 158 - 170 .

江巨浪 , 张佑生 , 薛峰 , 等 . 保持图像亮度的局部直方图均衡算法 [J]. 电子学报 , 2006 , 34 ( 5 ): 861 - 866 .

JIANG J L , ZHANG Y S , XUE F , et al . Local histogram equalization with brightness preservation [J]. Acta Electronica Sinica , 2006 , 34 ( 5 ): 861 - 866 . (in Chinese)

LAND E H , MCCANN J J . Lightness and retinex theory [J]. Journal of the Optical Society of America , 1971 , 61 ( 1 ): 1 - 11 .

JOBSON D J , RAHMAN Z , WOODELL G A . A multiscale retinex for bridging the gap between color images and the human observation of scenes [J]. IEEE Transactions on Image Processing , 1997 , 6 ( 7 ): 965 - 976 .

LEE C H , SHIH J L , LIEN C C , et al . Adaptive multiscale retinex for image contrast enhancement [C]// 2013 International Conference on Signal-Image Technology & Internet-Based Systems . Piscataway : IEEE , 2013 : 43 - 50 .

WEI C , WANG W J , YANG W H , et al . Deep retinex decomposition for low-light enhancement [C]// Proceedings of the Conference on British Machine Vision Conference . Newcastle : BMVA , 2018 : 1 - 12 .

HAI J , XUAN Z , YANG R , et al . R2RNet: Low-light image enhancement via real-low to real-normal network [J]. Journal of Visual Communication and Image Representation , 2023 , 90 : 103712 - 103720 .

JIANG Y F , GONG X Y , LIU D , et al . EnlightenGAN: Deep light enhancement without paired supervision [J]. IEEE Transactions on Image Processing , 2021 , 30 : 2340 - 2349 .

PRIYADARSHINI R , BHARANI A , RAHIMANKHAN E , et al . Low-light image enhancement using deep convolutional network [M]// Innovative Data Communication Technologies and Application . Berlin : Springer , 2021 : 695 - 705 .

LV F F , LU F , WU J H , et al . MBLLEN: Low-light image/video enhancement using CNNs [C]// Proceedings of the Conference on British Machine Vision Conference . Newcastle : BMVA , 2018 : 1 - 13 .

LIM S , KIM W . DSLR: Deep stacked Laplacian restorer for low-light image enhancement [J]. IEEE Transactions on Multimedia , 2021 , 23 : 4272 - 4284 .

ZHANG Y H , GUO X J , MA J Y , et al . Beyond brightening low-light images [J]. International Journal of Computer Vision , 2021 , 129 ( 4 ): 1013 - 1037 .

LIU C X , WU F D , WANG X . EFINet: Restoration for low-light images via enhancement-fusion iterative network [J]. IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 12 ): 8486 - 8499 .

WANG Y F , WAN R J , YANG W H , et al . Low-light image enhancement with normalizing flow [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Menlo Park : AAAI , 2022 , 36 ( 3 ): 2604 - 2612 .

FAN C M , LIU T J , LIU K H . Half wavelet attention on M-Net+ for low-light image enhancement [C]// 2022 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2022 : 3878 - 3882 .

LORE K G , AKINTAYO A , SARKAR S . LLNet: A deep autoencoder approach to natural low-light image enhancement [J]. Pattern Recognition , 2017 , 61 : 650 - 662 .

REN W Q , LIU S F , MA L , et al . Low-light image enhancement via a deep hybrid network [J]. IEEE Transactions on Image Processing , 2019 , 28 ( 9 ): 4364 - 4375 .

YAN Z C , ZHANG H , WANG B Y , et al . Automatic photo adjustment using deep neural networks [J]. ACM Transactions on Graphics , 2016 , 35 ( 2 ): 1 - 15 .

KIM H , CHOI S M , KIM C S , et al . Representative color transform for image enhancement [C]// 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2021 : 4459 - 4468 .

黄淑英 , 胡威 , 杨勇 , 等 . 基于渐进式双网络模型的低曝光图像增强方法 [J]. 计算机学报 , 2021 , 44 ( 2 ): 384 - 394 .

HUANG S Y , HU W , YANG Y , et al . A low-exposure image enhancement based on progressive dual network model [J]. Chinese Journal of Computers , 2021 , 44 ( 2 ): 384 - 394 . (in Chinese)

江泽涛 , 伍旭 , 张少钦 . 一种基于MR-VAE的低照度图像增强方法 [J]. 计算机学报 , 2020 , 43 ( 7 ): 1328 - 1339 .

JIANG Z T , WU X , ZHANG S Q . Low-illumination image enhancement based on MR-VAE [J]. Chinese Journal of Computers , 2020 , 43 ( 7 ): 1328 - 1339 . (in Chinese)

YANG W H , WANG S Q , FANG Y M , et al . From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 3063 - 3072 .

LIU X D , GAO Z , CHEN B M . MLFcGAN: Multilevel feature fusion-based conditional GAN for underwater image color correction [J]. IEEE Geoscience and Remote Sensing Letters , 2020 , 17 ( 9 ): 1488 - 1492 .

ZAMIR S W , ARORA A , KHAN S , et al . Learning enriched features for real image restoration and enhancement [M]// Computer Vision—ECCV 2020 . Cham : Springer International Publishing , 2020 : 492 - 511 .

CHEN L Y , LU X , ZHANG J , et al . HINet: Half instance normalization network for image restoration [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2021 : 182 - 192 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [M]// Computer Vision—ECCV 2018 . Cham : Springer International Publishing , 2018 : 3 - 19 .

SU Z H , ZHANG H S , LIU Z W , et al . Image deblurring algorithm by using conditional generation adversarial network [C]// 2021 40th Chinese Control Conference (CCC) . Piscataway : IEEE , 2021 : 8128 - 8133 .

IOFFE S , SZEGEDY C . Batch normalization: Accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International Conference on Machine Learning . Lille : PMLR , 2015 : 448 - 456 .

MAO X T , LIU Y M , LIU F Z , et al . Intriguing findings of frequency selection for image deblurring [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Menlo Park : AAAI , 2023 , 37 ( 2 ): 1905 - 1913 .

TALEBI H , MILANFAR P . Learning to resize images for computer vision tasks [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 497 - 506 .

JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution [M]// Computer Vision—ECCV 2016 . Cham : Springer International Publishing , 2016 : 694 - 711 .

PASZKE A , GROSS S , MASSA F , et al . PyTorch: An imperative style, high-performance deep learning library [EB/OL]. ( 2019-12-03 )[ 2023-03-01 ]. https://arxiv.org/abs/1912.01703 https://arxiv.org/abs/1912.01703 .

KINGMA D P , BA J . Adam: A method for stochastic optimization [EB/OL]. ( 2014-12-22 )[ 2023-03-01 ]. https://arxiv.org/abs/1412.6980 https://arxiv.org/abs/1412.6980 .

LOSHCHILOV I , HUTTER F . SGDR: Stochastic gradient descent with warm restarts [C]// International Conference on Learning Representations . Toulon : Openreview.net , 2017 : 163 - 178 .

BYCHKOVSKY V , PARIS S , CHAN E , et al . Learning photographic global tonal adjustment with a database of input/output image pairs [C]// 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2011 : 97 - 104 .

LI C Y , GUO C , HAN L H , et al . Low-light image and video enhancement using deep learning: A survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 12 ): 9396 - 9416 .

ZHU A Q , ZHANG L , SHEN Y , et al . Zero-shot restoration of underexposed images via robust retinex decomposition [C]// 2020 IEEE Conference on International Conference on Multimedia and Expo (ICME) . Piscataway : IEEE , 2020 : 1 - 6 .

LIANG J X , XU Y , QUAN Y H , et al . Self-supervised low-light image enhancement using discrepant untrained network priors [J]. IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 11 ): 7332 - 7345 .

LIU R S , MA L , ZHANG J A , et al . Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 10561 - 10570 .

LI C Y , GUO C L , LOY C C . Learning to enhance low-light image via zero-reference deep curve estimation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 8 ): 4225 - 4238 .

LI C Y , GUO C L , REN W Q , et al . An underwater image enhancement benchmark dataset and beyond [J]. IEEE Transactions on Image Processing , 2019 , 29 : 4376 - 4389 .

LI C Y , ANWAR S , HOU J H , et al . Underwater image enhancement via medium transmission-guided multi-color space embedding [J]. IEEE Transactions on Image Processing , 2021 , 30 : 4985 - 5000 .

BERMAN D , LEVY D , AVIDAN S , et al . Underwater single image color restoration using haze-lines and a new quantitative dataset [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 8 ): 2822 - 2837 .

PANETTA K , GAO C , AGAIAN S . Human-visual-system-inspired underwater image quality measures [J]. IEEE Journal of Oceanic Engineering , 2016 , 41 ( 3 ): 541 - 551 .

ANCUTI C , ANCUTI C O , HABER T , et al . Enhancing underwater images and videos by fusion [C]// 2012 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2012 : 81 - 88 .

WANG S H , ZHENG J , HU H M , et al . Naturalness preserved enhancement algorithm for non-uniform illumination images [J]. IEEE Transactions on Image Processing , 2013 , 22 ( 9 ): 3538 - 3548 .

LI C Y , ANWAR S , PORIKLI F . Underwater scene prior inspired deep underwater image and video enhancement [J]. Pattern Recognition , 2020 , 98 : 107038 .

PENG L , ZHU C , BIAN L . U-shape transformer for underwater image enhancement [J]. IEEE Transactions on Image Processing , 2023 , 32 : 3066 - 3079 .

REN T D , XU H Y , JIANG G , et al . Reinforced swin-convs transformer for underwater image enhancement [J]. IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 1 - 16 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image Classification Network of Gating Mechanism

Few-Shot SAR Target Recognition Method Based on Feature Fusion Attention and Meta-ResNet

Intrusion Detection Method Based on Capsule Network for Industrial Internet

Lightweight and Low Complexity CSI Feedback Method for FDD Massive MIMO Systems

Related Author

LIU Wan-jun

YUAN Heng

GAO Yuan

JIANG Wen-tao

LIU Qi

LIU Yong-xiang

ZHANG Xin-yu

HU Xiang-dong

Related Institution

School of Software, Liaoning Technical University

College of Electronic Science and Technology， National University of Defense Technology

School of Automation/School of Industrial Internet, Chongqing University of Posts and Telecommunications

School of Microelectronics and Communication Engineering, Chongqing University

School of Artificial Intelligence, Chongqing University of Technology

⁰