一种利用改进深度图像先验构建的图像降噪模型

doi:10.12263/DZXB.20211323

电子学报 ›› 2022, Vol. 50 ›› Issue (7): 1573-1578.DOI: 10.12263/DZXB.20211323

一种利用改进深度图像先验构建的图像降噪模型

徐少平(), 李芬, 陈孝国, 陈晓军, 江顺亮

南昌大学数学与计算机学院，江西南昌 330031

收稿日期:2021-09-28 修回日期:2022-03-06 出版日期:2022-07-25 发布日期:2022-07-30
通讯作者: 徐少平
作者简介:徐少平男，1976年5月出生于江西省九江市．博士，南昌大学数学与计算机学院计算机科学与技术系教授，博士生导师.主要研究方向为图形图像处理、机器视觉、虚拟手术仿真等．
李芬女，1995年8月出生于江西省南昌市. 现为南昌大学硕士研究生，主要研究方向为图像处理与计算机视觉．E-mail: 411014519034@email.ncu.edu.cn
基金资助:
国家自然科学基金(62162043)

An Image Denoising Model Using the Improved Deep Image Prior

XU Shao-ping(), LI Fen, CHEN Xiao-guo, CHEN Xiao-jun, JIANG Shun-liang

School of Mathematics and Computer Sciences，Nanchang University，Nanchang，Jiangxi 330031，China

Received:2021-09-28 Revised:2022-03-06 Online:2022-07-25 Published:2022-07-30
Contact: XU Shao-ping

摘要/Abstract

摘要：

为进一步提高深度图像先验（Deep Image Prior，DIP）降噪模型的降噪效果和执行效率，从网络结构、网络输入和Loss函数三个方面对其进行改进从而获得了一种改进的深度图像先验（Improved Deep Image Prior，IDIP）降噪模型.具体地，在网络结构方面，通过新增非线性特征传递路径的方法将原DIP模型编码器-解码器（encoder-decoder）架构中相同尺度特征层之间所采用的简单连接改进为复杂连接，有利于特征信息调制与传递从而提高神经网络的非线性映射能力；在网络输入方面，用已具有较高图像质量的初步降噪图像替换随机张量作为网络输入向网络模型提供更为丰富的信息，有利于加快网络收敛速度进而提高执行效率；在Loss函数方面，在原噪声图像的基础上新增初步降噪图像作为第二目标图像，有利于提高Loss函数的导向能力从而提高降噪效果.实验结果表明：所提出的IDIP降噪模型在各噪声水平值下的降噪性能和执行效率均显著优于原DIP模型；与现有的主流降噪方法相比，IDIP降噪模型也具有更好的降噪效果.

关键词: 深度图像先验, 降噪效果, 执行效率, 复杂连接, 混合Loss

Abstract:

To further improve the execution efficiency and denoising effect of the deep image prior(DIP) denoising model, an improved deep image prior(IDIP) denoising model was proposed by improving the original DIP from three aspects, network architecture, network input, and Loss function. Specifically, considering the network architecture, the simple connection adopted in the encoder-decoder backbone network was promoted with complex connection by adding nonlinear features transferring path,which brings benefit to the information modulation and transmission of features between encoder and decoder at the same level. In aspect of the network input, the random tensor was replaced by the preliminary denoised image with better image quality, and the preliminary denoised image can provide more abundant information to the network model, accelerating the convergence speed of network and improving the execution efficiency. Regarding the Loss function, the preliminary denoised image was added as the second target image to improve the guidance ability of the loss function, improving denoising effect significantly. Extensive experiments show that, the proposed IDIP denoising model significantly outperforms original one at various noise levels in terms of denoising effect and execution efficiency, and it also has a better performance than other state-of-the-art methods with regard to denoising effect.

Key words: deep image prior, denoising effect, execution efficiency, complex connection, hybrid loss

中图分类号:

TP391.4

徐少平, 李芬, 陈孝国, 陈晓军, 江顺亮. 一种利用改进深度图像先验构建的图像降噪模型[J]. 电子学报, 2022, 50(7): 1573-1578.

XU Shao-ping, LI Fen, CHEN Xiao-guo, CHEN Xiao-jun, JIANG Shun-liang. An Image Denoising Model Using the Improved Deep Image Prior[J]. Acta Electronica Sinica, 2022, 50(7): 1573-1578.

图/表 8

图1 IDIP降噪模型框图

表1 骨干网络的改进对降噪性能的影响(dB)

模型	House	Peppers	Barbara	Boat	Cameraman	Couple	Hill	Lena	Man	Monarch	平均值	标准差
DIP	30.84	28.00	24.38	27.31	26.81	27.25	27.89	29.99	27.56	27.27	27.73	1.66
IDIP-1	30.92	28.09	24.79	27.43	26.96	27.26	28.00	30.03	27.61	26.82	27.79	1.61

表2 网络输入和Loss函数的改进对降噪性能的影响(dB)

模型	House	Peppers	Barbara	Boat	Cameraman	Couple	Hill	Lena	Man	Monarch	平均值	标准差
DIP	30.84	28.00	24.38	27.31	26.81	27.25	27.89	29.99	27.56	27.27	27.73	1.66
FFDNet	32.37	29.63	28.70	29.46	28.73	29.22	29.38	31.87	29.11	28.70	29.72	1.25
IDIP-2	32.40	29.59	29.09	29.65	28.40	29.32	29.46	32.06	29.34	28.75	29.80	1.27
IDIP	32.99	30.34	29.40	30.18	29.24	29.76	29.91	32.56	29.67	29.45	30.35	1.26

表3 在噪声水平σ=30下不同初步降噪图像对降噪性能的影响(dB)

模型	House	Peppers	Barbara	Boat	Cameraman	Couple	Hill	Lena	Man	Monarch	平均值
IDIP(DnCNN)	32.91	30.41	29.30	30.13	29.23	29.61	29.98	32.58	29.69	29.59	30.34
IDIP(BM3D)	32.63	30.12	29.83	29.86	29.06	29.46	29.82	32.25	29.38	29.33	30.17
IDIP(NCSR)	32.87	29.92	29.82	29.84	29.16	29.40	29.80	32.19	29.44	29.39	30.18
IDIP(WNNM)	33.07	30.16	30.26	30.07	29.14	29.51	29.96	32.28	29.51	29.38	30.33
IDIP(FFDNet)	32.99	30.34	29.40	30.18	29.24	29.76	29.91	32.56	29.67	29.45	30.35

图2 各对比方法在Lena图像上的降噪效果对比

表4 各对比方法在常用图像上所获得的PSNR均值比较（dB）

$σ$	10	20	30	40	50	60	平均值
FFDNet	34.83	31.40	29.72	28.50	27.60	26.85	29.82
BM3D	34.82	31.44	29.59	28.13	27.54	26.37	29.65
NCSR	34.81	31.38	29.43	28.06	27.02	26.08	29.46
WNNM	34.94	31.61	29.80	28.48	27.51	26.68	29.84
DnCNN	34.94	31.69	29.83	28.52	27.56	26.65	29.87
DIP	32.41	29.32	27.73	26.60	25.56	24.77	27.73
IDIP	35.68	32.13	30.35	29.16	28.13	27.22	30.45

表4 各对比方法在常用图像上所获得的PSNR均值比较（dB）

$σ$	10	20	30	40	50	60	平均值
FFDNet	34.83	31.40	29.72	28.50	27.60	26.85	29.82
BM3D	34.82	31.44	29.59	28.13	27.54	26.37	29.65
NCSR	34.81	31.38	29.43	28.06	27.02	26.08	29.46
WNNM	34.94	31.61	29.80	28.48	27.51	26.68	29.84
DnCNN	34.94	31.69	29.83	28.52	27.56	26.65	29.87
DIP	32.41	29.32	27.73	26.60	25.56	24.77	27.73
IDIP	35.68	32.13	30.35	29.16	28.13	27.22	30.45

表5 各对比方法在50张BSD图像上所获得的PSNR均值比较(dB)

$σ$	10	20	30	40	50	60	平均值
FFDNet	33.61	30.02	28.16	26.93	26.04	25.34	28.25
BM3D	33.41	29.56	27.60	26.26	25.40	24.73	27.83
NCSR	33.42	29.58	27.60	26.27	25.35	24.59	27.80
WNNM	33.53	29.73	27.81	26.54	25.63	24.92	28.03
DnCNN	33.69	30.06	28.14	26.87	25.99	25.23	28.33
DIP	31.27	27.97	26.22	25.09	24.18	23.43	26.36
IDIP	35.02	30.80	28.84	27.56	26.55	25.76	29.09

表5 各对比方法在50张BSD图像上所获得的PSNR均值比较(dB)

$σ$	10	20	30	40	50	60	平均值
FFDNet	33.61	30.02	28.16	26.93	26.04	25.34	28.25
BM3D	33.41	29.56	27.60	26.26	25.40	24.73	27.83
NCSR	33.42	29.58	27.60	26.27	25.35	24.59	27.80
WNNM	33.53	29.73	27.81	26.54	25.63	24.92	28.03
DnCNN	33.69	30.06	28.14	26.87	25.99	25.23	28.33
DIP	31.27	27.97	26.22	25.09	24.18	23.43	26.36
IDIP	35.02	30.80	28.84	27.56	26.55	25.76	29.09

表6 DIP与IDIP降噪模型执行迭代次数和执行时间的比较

图像	Peppers	Barbara	Boat	Hill	Lena
PSNR	26.67dB	23.37dB	26.34dB	26.94dB	29.29dB
DIP	1 048(117.80s)	2 132(466.80s)	2 041(444.95s)	1 802(397.20s)	1 775(387.42s)
IDIP	245(34.38s)	159(40.25s)	203(48.55s)	245(59.40s)	255(60.69s)

参考文献 13

1	LI J X, GUO X B, LU G M, et al. DRPL: Deep regression pair learning for multi-focus image fusion[J]. IEEE Transactions on Image Processing, 2020, 29(3): 4816-4831.
2	WANG C H, HOU B C, SHI J, et al. Uniformity evaluation of temperature field in an oven based on image processing[J]. IEEE Access, 2020, 8(1): 10243-10253.
3	ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
4	ZHANG K, ZUO W M, ZHANG L. FFDNet: Toward a fast and flexible solution for CNN based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4608-4622.
5	YUE Z S, YONG H W, ZHAO Q, et al. Variational denoising network: Toward blind noise modeling and removal[C]//Advances in Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2019: 1-12.
6	DMITRY U, VEDALDI A, VICTOR L. Deep image prior[J]. International Journal of Computer Vision, 2020, 128(8): 1867-1888.
7	VIGNESH R, BALAMURALI M, SHARATH M S, et al. Deep network for capacitive ECG denoising[C]//Medical Measurements and Applications. Istanbul: IEEE, 2019:1-6.
8	DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080-2095.
9	FU J, DONG J B, ZHAO F. A deep learning reconstruction framework for differential phase-contrast computed tomography with incomplete data[J]. IEEE Transactions on Image Processing, 2020, 29(1): 2190-2202.
10	ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 898-916.
11	DONG W S, ZHANG L, SHI G M, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4): 1620-1630.
12	GU S H, ZHANG L, ZUO W M, et al. Weighted nuclear norm minimization with application to image denoising[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Columbus: IEEE Computer Society, 2014: 2862-2869.
13	SOH J W, CHO S, CHO N I. Meta-transfer learning for zero-shot super-resolution[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Seattle: IEEE Computer Society, 2020: 3513-3522.

一种利用改进深度图像先验构建的图像降噪模型

An Image Denoising Model Using the Improved Deep Image Prior

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 13

相关文章 1

编辑推荐

Metrics