Dual-Channel Deep Image Prior for Image Denoising

XU Shao-ping; XIAO Nan; LUO Jie; CHENG Xiao-hui; CHEN Xiao-jun

doi:10.12263/DZXB.20221012

您当前的位置：

首页 >

文章列表页 >

Dual-Channel Deep Image Prior for Image Denoising

PAPERS | 更新时间：2026-04-10

- Dual-Channel Deep Image Prior for Image Denoising
- ACTA ELECTRONICA SINICA Vol. 52, Issue 1, Pages: 58-68(2024)
- 作者机构：
  
  1.南昌大学数学与计算机学院，江西南昌 330031
  2.南昌大学附属感染病医院，江西南昌 330006
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20221012
  CLC： TP391.4;
- Received：02 September 2022，
  
  Revised：2022-12-27，
  
  Published：25 January 2024
- 稿件说明：
移动端阅览
徐少平,肖楠,罗洁,等.双通道深度图像先验降噪模型[J].电子学报,2024,52(01):58-68.

XU Shao-ping, XIAO Nan, LUO Jie, et al.Dual-Channel Deep Image Prior for Image Denoising[J].Acta Electronica Sinica, 2024, 52(01): 58-68.
徐少平,肖楠,罗洁,等.双通道深度图像先验降噪模型[J].电子学报,2024,52(01):58-68. DOI：10.12263/DZXB.20221012

XU Shao-ping, XIAO Nan, LUO Jie, et al.Dual-Channel Deep Image Prior for Image Denoising[J].Acta Electronica Sinica, 2024, 52(01): 58-68. DOI：10.12263/DZXB.20221012

摘要

相对于采用固定网络参数值的有监督深度降噪模型而言，无监督的深度图像先验（Deep Image Prior，DIP）降噪模型更具灵活性和实用性.然而，DIP模型的降噪效果远低于有监督降噪模型（尤其是在处理人工合成噪声图像时）.为进一步提升DIP降噪模型的降噪效果，本文提出了双通道深度图像先验降噪模型.该降噪模型由噪声图像预处理、在线迭代训练和图像融合3个模块组成.首先，利用BM3D和FFDNet两种经典降噪方法对给定的噪声图像进行预处理，得到2张初步降噪图像，然后，将原DIP单通道逼近目标图像架构拓展为双通道工作模式.其中，第一通道以FFDNet初步降噪图像和噪声图像为双目标图像，第二通道则以BM3D预处理图像和噪声图像为双目标图像.在此基础上，按照标准的DIP在线训练方式让DIP网络输出图像在两个通道上分别逼近各自的目标图像，同时依据基于边缘能量定义的伪有参考图像质量评价值适时终止迭代过程，从而获得2张中间生成图像.最后，使用结构化图块分解融合算法将两张中间生成图像融合并作为最终的降噪后图像.实验数据表明，在合成噪声图像上，本文提出的双通道深度图像先验降噪模型在各个噪声水平上显著优于原DIP及其他无监督降噪模型（提升了约2.2 dB），甚至逼近和超过了新近提出的主流有监督降噪模型，这充分表明了本文提出的改进策略的有效性；在真实噪声图像上，本文提出的降噪模型优于排名第二的对比降噪方法约2 dB，展现出其在实际应用场景下独有的优势.

Abstract

Compared with the supervised deep learning-based denoising models adopting fixed network parameter values

the unsupervised deep image prior (DIP) is more flexible and practical than those supervised denoising models. However

the overall performance of the unsupervised DIP model is far lower than those supervised ones

especially when it is easy to obtain training data such as synthetic noisy images. To improve the performance of the DIP model

in this paper we propose a denoising model called dual-channel deep image prior (DCDIP). The DCDIP model consists of three modules: preprocessing

online training

and image fusion. First

two classical denoising methods

i.e.

BM3D and FFDNet

are used to preprocess a given noisy image to obtain two corresponding initial denoised images. Then

the original DIP model with single channel approximation target image architecture is expanded to a dual-channel working manner. In the first channel

the initial denoised image obtained with FFDNet and the noisy image are taken as the dual-target images. Similarly

in the second channel

the initial denoised image obtained with BM3D and the noisy image are taken as the dual-target images. On this basis

according to the original DIP online training manner

the output image of the DCDIP is iteratively trained to approach the respective target images in the two channels

and the iterative process is terminated in time with the proposed pseudo reference image quality evaluation index based on the edge energy. As such

we can obtain two intermediate generated images with high quality. Finally

the two intermediate generated images are fused as the final denoised image by using the structural patch decomposition (SPD) fusion algorithm. The experimental results show that the proposed DCDIP significantly outperforms the original DIP model and unsupervised ones by about 2.2 dB across different noise levels. In addition

it even approaches and surpasses the recently proposed supervised denoising models

demonstrating the effectiveness of our improvement strategy. On the real-world noisy image

the proposed DCDIP outperforms the second-ranked competing denoising method by about 2 dB

which verifies its unique advantages in the practical application scenarios. The performance improvement is mainly due to the hybrid use of internal and external image prior-based denoising methods under the dual-channel DIP framework.

关键词

Keywords

references

JIA X X , LIU S Y , FENG X C , et al . FOCNet: A fractional optimal control network for image denoising [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 6047 - 6056 .

GUO S , YAN Z F , ZHANG K , et al . Toward convolutional blind denoising of real photographs [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 1712 - 1722 .

CHANG M , LI Q , FENG H J , et al . Spatial-adaptive network for single image denoising [C]// 2020 European Conference on Computer Vision (ECCV) . Cham : Springer , 2020 : 171 - 187 .

REN C , HE X H , WANG C C , et al . Adaptive consistency prior based deep network for image denoising [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 8592 - 8602 .

XU J , ZHANG L , ZHANG D . External prior guided internal prior learning for real-world noisy image denoising [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 6 ): 2996 - 3010 .

盖杉 , 鲍中运 . 基于深度学习的高噪声图像去噪算法 [J]. 自动化学报 , 2020 , 46 ( 12 ): 2672 - 2680 .

GAI S , BAO Z Y . High noise image denoising algorithm based on deep learning [J]. Acta Automatica Sinica , 2020 , 46 ( 12 ): 2672 - 2680 . (in Chinese)

ZHANG K , ZUO W M , CHEN Y J , et al . Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 7 ): 3142 - 3155 .

ZHANG K , ZUO W M , ZHANG L . FFDNet: Toward a fast and flexible solution for CNN based image denoising [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 9 ): 4608 - 4622 .

ZHANG K , LI Y W , ZUO W M , et al . Plug-and-play image restoration with deep denoiser prior [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 10 ): 6360 - 6376 .

LIANG J Y , CAO J Z , SUN G L , et al . SwinIR: Image restoration using swin transformer [C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Piscataway : IEEE , 2021 : 1833 - 1844 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer: Efficient transformer for high-resolution image restoration [EB/OL]. ( 2021-11-18 )[ 2022-08-29 ]. https://arxiv.org/abs/2111.09881 https://arxiv.org/abs/2111.09881 .

ULYANOV D , VEDALDI A , LEMPITSKY V . Deep image prior [J]. International Journal of Computer Vision , 2020 , 128 ( 7 ): 1867 - 1888 .

DABOV K , FOI A , KATKOVNIK V , et al . Image denoising by sparse 3-D transform-domain collaborative filtering [J]. IEEE Transactions on Image Processing , 2007 , 16 ( 8 ): 2080 - 2095 .

MA K D , LI H , YONG H W , et al . Robust multi-exposure image fusion: A structural patch decomposition approach [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 5 ): 2519 - 2532 .

MEINHARDT T , MOELLER M , HAZIRBAS C , et al . Learning proximal operators: Using denoising networks for regularizing inverse imaging problems [C]// 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 1799 - 1808 .

CHOI J H , ELGENDY O A , CHAN S H . Optimal combination of image denoisers [J]. IEEE Transactions on Image Processing , 2019 , 28 ( 8 ): 4016 - 4031 .

YU L J , LUO J , XU S P , al el . An unsupervised weight map generative network for pixel-level combination of image denoisers [J]. Applied. Sciences , 2022 , 12 ( 12 ): 6227 .

LUO J Y , XU S P , LI C X . A fast denoising fusion network using internal and external priors [J]. Signal, Image and Video Processing , 2021 , 15 ( 6 ): 1275 - 1283 .

VU P V , CHANDLER D M . A fast wavelet-based algorithm for global and local image sharpness estimation [J]. IEEE Signal Processing Letters , 2012 , 19 ( 7 ): 423 - 426 .

周先春 , 吴婷 , 石兰芳 , 等 . 一种基于曲率变分正则化的小波变换图像去噪方法 [J]. 电子学报 , 2018 , 46 ( 3 ): 621 - 628 .

ZHOU X C , WU T , SHI L F , et al . A kind of wavelet transform image denoising method based on curvature variation regularization [J]. Acta Electronica Sinica , 2018 , 46 ( 3 ): 621 - 628 . (in Chinese)

WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment: From error visibility to structural similarity [J]. IEEE Transactions on Image Processing , 2004 , 13 ( 4 ): 600 - 612 .

ZHANG K , ZUO W M , GU S H , et al . Learning deep CNN denoiser prior for image restoration [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 2808 - 2817 .

MOU C , ZHANG J , WU Z Y . Dynamic attentive graph learning for image restoration [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4308 - 4317 .

KRULL A , BUCHHOLZ T O , JUG F . Noise2Void - Learning denoising from single noisy images [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 2124 - 2132 .

MARTIN D , FOWLKES C , TAL D , et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings Eighth IEEE International Conference on Computer Vision . Piscataway : IEEE , 2001 : 416 - 423 .

NAM S , HWANG Y , MATSUSHITA Y , et al . A holistic approach to cross-channel image noise modeling and its application to image denoising [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 1683 - 1691 .

YUE Z S , YONG H W , ZHAO Q , et al . Variational denoising network: Toward blind noise modeling and removal [C]// Advances in Neural Information Processing Systems . Vancouver : NeurIPS , 2019 : 1 - 12 .

ZHANG K , XIE M Y , GOR M , et al . Metadip: Accelerating deep image prior with meta-learning [EB/OL]. ( 2022-09-18 )[ 2022-12-27 ]. https://arxiv.org/abs/2209.08452 https://arxiv.org/abs/2209.08452 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Malware Detection Method Based on Multi-Dimensional Dynamic Weighted Alpha Image Fusion and Feature Enhancement

A Scene-Adaptive Dual-Branch Efficient Cattle Facial Recognition Algorithm

MW-DAN: Multilevel Wavelet-Deep Aggregation Network for Hyperspectral Image Super-Resolution

A Kernel Method for PET Image Reconstruction with Graph Laplacian Regularization

An Image Denoising Model Using the Improved Deep Image Prior

Related Author

XIE Li-xia

WEI Chen-yang

YANG Hong-yu

HU Ze

CHENG Xiang

JIAO Jie

QI Yong-sheng

LIU Li-qiang

Related Institution

School of Computer Science and Technology, Civil Aviation University of China

School of Safety Science and Engineering, Civil Aviation University of China

School of Information Engineering, Yangzhou University

Key Laboratory of Civil Aviation Flight Networking, Civil Aviation University of China

School of Electric Power, Inner Mongolia University of Technology

⁰