双通道深度图像先验降噪模型

徐少平; 肖楠; 罗洁; 程晓慧; 陈晓军

doi:10.12263/DZXB.20221012

您当前的位置：

首页 >

文章列表页 >

双通道深度图像先验降噪模型

学术论文 | 更新时间：2026-04-10

- 双通道深度图像先验降噪模型
- Dual-Channel Deep Image Prior for Image Denoising
- 电子学报 2024年52卷第1期页码：58-68
- 作者机构：
  
  1.南昌大学数学与计算机学院，江西南昌 330031
  2.南昌大学附属感染病医院，江西南昌 330006
- 作者简介：
  
  [ "徐少平男，1976年5月生，江西九江人．博士.南昌大学数学与计算机学院计算机科学与技术系教授.博士生导师.主要研究方向为图形图像处理、机器视觉、虚拟手术仿真等． E-mail: xushaoping@ncu.edu.cn" ]
  [ "肖楠男，1999年7月生，江西吉安人.现为南昌大学硕士研究生.主要研究方向为图像处理与深度学习． E-mail: 406100210085@email.ncu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62162043)
- DOI：10.12263/DZXB.20221012
  中图分类号： TP391.4;
- 收稿：2022-09-02，
  
  修回：2022-12-27，
  
  纸质出版：2024-01-25
- 稿件说明：
移动端阅览
徐少平,肖楠,罗洁,等.双通道深度图像先验降噪模型[J].电子学报,2024,52(01):58-68.

XU Shao-ping, XIAO Nan, LUO Jie, et al.Dual-Channel Deep Image Prior for Image Denoising[J].Acta Electronica Sinica, 2024, 52(01): 58-68.
徐少平,肖楠,罗洁,等.双通道深度图像先验降噪模型[J].电子学报,2024,52(01):58-68. DOI：10.12263/DZXB.20221012

XU Shao-ping, XIAO Nan, LUO Jie, et al.Dual-Channel Deep Image Prior for Image Denoising[J].Acta Electronica Sinica, 2024, 52(01): 58-68. DOI：10.12263/DZXB.20221012

摘要

相对于采用固定网络参数值的有监督深度降噪模型而言，无监督的深度图像先验（Deep Image Prior，DIP）降噪模型更具灵活性和实用性.然而，DIP模型的降噪效果远低于有监督降噪模型（尤其是在处理人工合成噪声图像时）.为进一步提升DIP降噪模型的降噪效果，本文提出了双通道深度图像先验降噪模型.该降噪模型由噪声图像预处理、在线迭代训练和图像融合3个模块组成.首先，利用BM3D和FFDNet两种经典降噪方法对给定的噪声图像进行预处理，得到2张初步降噪图像，然后，将原DIP单通道逼近目标图像架构拓展为双通道工作模式.其中，第一通道以FFDNet初步降噪图像和噪声图像为双目标图像，第二通道则以BM3D预处理图像和噪声图像为双目标图像.在此基础上，按照标准的DIP在线训练方式让DIP网络输出图像在两个通道上分别逼近各自的目标图像，同时依据基于边缘能量定义的伪有参考图像质量评价值适时终止迭代过程，从而获得2张中间生成图像.最后，使用结构化图块分解融合算法将两张中间生成图像融合并作为最终的降噪后图像.实验数据表明，在合成噪声图像上，本文提出的双通道深度图像先验降噪模型在各个噪声水平上显著优于原DIP及其他无监督降噪模型（提升了约2.2 dB），甚至逼近和超过了新近提出的主流有监督降噪模型，这充分表明了本文提出的改进策略的有效性；在真实噪声图像上，本文提出的降噪模型优于排名第二的对比降噪方法约2 dB，展现出其在实际应用场景下独有的优势.

Abstract

Compared with the supervised deep learning-based denoising models adopting fixed network parameter values

the unsupervised deep image prior (DIP) is more flexible and practical than those supervised denoising models. However

the overall performance of the unsupervised DIP model is far lower than those supervised ones

especially when it is easy to obtain training data such as synthetic noisy images. To improve the performance of the DIP model

in this paper we propose a denoising model called dual-channel deep image prior (DCDIP). The DCDIP model consists of three modules: preprocessing

online training

and image fusion. First

two classical denoising methods

i.e.

BM3D and FFDNet

are used to preprocess a given noisy image to obtain two corresponding initial denoised images. Then

the original DIP model with single channel approximation target image architecture is expanded to a dual-channel working manner. In the first channel

the initial denoised image obtained with FFDNet and the noisy image are taken as the dual-target images. Similarly

in the second channel

the initial denoised image obtained with BM3D and the noisy image are taken as the dual-target images. On this basis

according to the original DIP online training manner

the output image of the DCDIP is iteratively trained to approach the respective target images in the two channels

and the iterative process is terminated in time with the proposed pseudo reference image quality evaluation index based on the edge energy. As such

we can obtain two intermediate generated images with high quality. Finally

the two intermediate generated images are fused as the final denoised image by using the structural patch decomposition (SPD) fusion algorithm. The experimental results show that the proposed DCDIP significantly outperforms the original DIP model and unsupervised ones by about 2.2 dB across different noise levels. In addition

it even approaches and surpasses the recently proposed supervised denoising models

demonstrating the effectiveness of our improvement strategy. On the real-world noisy image

the proposed DCDIP outperforms the second-ranked competing denoising method by about 2 dB

which verifies its unique advantages in the practical application scenarios. The performance improvement is mainly due to the hybrid use of internal and external image prior-based denoising methods under the dual-channel DIP framework.

关键词

Keywords

references

JIA X X , LIU S Y , FENG X C , et al . FOCNet: A fractional optimal control network for image denoising [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 6047 - 6056 .

GUO S , YAN Z F , ZHANG K , et al . Toward convolutional blind denoising of real photographs [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 1712 - 1722 .

CHANG M , LI Q , FENG H J , et al . Spatial-adaptive network for single image denoising [C]// 2020 European Conference on Computer Vision (ECCV) . Cham : Springer , 2020 : 171 - 187 .

REN C , HE X H , WANG C C , et al . Adaptive consistency prior based deep network for image denoising [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 8592 - 8602 .

XU J , ZHANG L , ZHANG D . External prior guided internal prior learning for real-world noisy image denoising [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 6 ): 2996 - 3010 .

盖杉 , 鲍中运 . 基于深度学习的高噪声图像去噪算法 [J]. 自动化学报 , 2020 , 46 ( 12 ): 2672 - 2680 .

GAI S , BAO Z Y . High noise image denoising algorithm based on deep learning [J]. Acta Automatica Sinica , 2020 , 46 ( 12 ): 2672 - 2680 . (in Chinese)

ZHANG K , ZUO W M , CHEN Y J , et al . Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 7 ): 3142 - 3155 .

ZHANG K , ZUO W M , ZHANG L . FFDNet: Toward a fast and flexible solution for CNN based image denoising [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 9 ): 4608 - 4622 .

ZHANG K , LI Y W , ZUO W M , et al . Plug-and-play image restoration with deep denoiser prior [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 10 ): 6360 - 6376 .

LIANG J Y , CAO J Z , SUN G L , et al . SwinIR: Image restoration using swin transformer [C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Piscataway : IEEE , 2021 : 1833 - 1844 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer: Efficient transformer for high-resolution image restoration [EB/OL]. ( 2021-11-18 )[ 2022-08-29 ]. https://arxiv.org/abs/2111.09881 https://arxiv.org/abs/2111.09881 .

ULYANOV D , VEDALDI A , LEMPITSKY V . Deep image prior [J]. International Journal of Computer Vision , 2020 , 128 ( 7 ): 1867 - 1888 .

DABOV K , FOI A , KATKOVNIK V , et al . Image denoising by sparse 3-D transform-domain collaborative filtering [J]. IEEE Transactions on Image Processing , 2007 , 16 ( 8 ): 2080 - 2095 .

MA K D , LI H , YONG H W , et al . Robust multi-exposure image fusion: A structural patch decomposition approach [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 5 ): 2519 - 2532 .

MEINHARDT T , MOELLER M , HAZIRBAS C , et al . Learning proximal operators: Using denoising networks for regularizing inverse imaging problems [C]// 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 1799 - 1808 .

CHOI J H , ELGENDY O A , CHAN S H . Optimal combination of image denoisers [J]. IEEE Transactions on Image Processing , 2019 , 28 ( 8 ): 4016 - 4031 .

YU L J , LUO J , XU S P , al el . An unsupervised weight map generative network for pixel-level combination of image denoisers [J]. Applied. Sciences , 2022 , 12 ( 12 ): 6227 .

LUO J Y , XU S P , LI C X . A fast denoising fusion network using internal and external priors [J]. Signal, Image and Video Processing , 2021 , 15 ( 6 ): 1275 - 1283 .

VU P V , CHANDLER D M . A fast wavelet-based algorithm for global and local image sharpness estimation [J]. IEEE Signal Processing Letters , 2012 , 19 ( 7 ): 423 - 426 .

周先春 , 吴婷 , 石兰芳 , 等 . 一种基于曲率变分正则化的小波变换图像去噪方法 [J]. 电子学报 , 2018 , 46 ( 3 ): 621 - 628 .

ZHOU X C , WU T , SHI L F , et al . A kind of wavelet transform image denoising method based on curvature variation regularization [J]. Acta Electronica Sinica , 2018 , 46 ( 3 ): 621 - 628 . (in Chinese)

WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment: From error visibility to structural similarity [J]. IEEE Transactions on Image Processing , 2004 , 13 ( 4 ): 600 - 612 .

ZHANG K , ZUO W M , GU S H , et al . Learning deep CNN denoiser prior for image restoration [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 2808 - 2817 .

MOU C , ZHANG J , WU Z Y . Dynamic attentive graph learning for image restoration [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4308 - 4317 .

KRULL A , BUCHHOLZ T O , JUG F . Noise2Void - Learning denoising from single noisy images [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 2124 - 2132 .

MARTIN D , FOWLKES C , TAL D , et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings Eighth IEEE International Conference on Computer Vision . Piscataway : IEEE , 2001 : 416 - 423 .

NAM S , HWANG Y , MATSUSHITA Y , et al . A holistic approach to cross-channel image noise modeling and its application to image denoising [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 1683 - 1691 .

YUE Z S , YONG H W , ZHAO Q , et al . Variational denoising network: Toward blind noise modeling and removal [C]// Advances in Neural Information Processing Systems . Vancouver : NeurIPS , 2019 : 1 - 12 .

ZHANG K , XIE M Y , GOR M , et al . Metadip: Accelerating deep image prior with meta-learning [EB/OL]. ( 2022-09-18 )[ 2022-12-27 ]. https://arxiv.org/abs/2209.08452 https://arxiv.org/abs/2209.08452 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于多维度动态加权alpha图像融合与特征增强的恶意软件检测方法

一种场景自适应的双分支牛脸高效识别算法

基于多层小波深度聚合网络的高光谱图像超分辨率方法

基于图拉普拉斯正则化的PET图像核重建方法

一种利用改进深度图像先验构建的图像降噪模型