Deep Prior-Based Blind Image Deblurring

BAI Yong-qiang; YU Jing; LI Yi-nong; XIAO Chuang-bai

doi:10.12263/DZXB.20211483

您当前的位置：

首页 >

文章列表页 >

Deep Prior-Based Blind Image Deblurring

PAPERS | 更新时间：2025-12-08

- Deep Prior-Based Blind Image Deblurring
- ACTA ELECTRONICA SINICA Vol. 51, Issue 4, Pages: 1050-1067(2023)
- 作者机构：
  
  北京工业大学信息学部，北京100124
- 作者简介：
  
  liyinong@emails.bjut.edu.cn
- 基金信息：
  
  Beijing Natural Science Foundation(4212014);Scientific Research Common Program of Beijing Municipal Commission of Education(KM201910005029)
- DOI：10.12263/DZXB.20211483
  CLC： TP391.41;
- Received：03 November 2021，
  
  Revised：2022-02-17，
  
  Published：25 April 2023
- 稿件说明：
移动端阅览
白勇强,禹晶,李一秾等.基于深度先验的盲图像去模糊算法[J].电子学报,2023,51(04):1050-1067.

BAI Yong-qiang,YU Jing,LI Yi-nong,et al.Deep Prior-Based Blind Image Deblurring[J].ACTA ELECTRONICA SINICA,2023,51(04):1050-1067.
白勇强,禹晶,李一秾等.基于深度先验的盲图像去模糊算法[J].电子学报,2023,51(04):1050-1067. DOI： 10.12263/DZXB.20211483.

BAI Yong-qiang,YU Jing,LI Yi-nong,et al.Deep Prior-Based Blind Image Deblurring[J].ACTA ELECTRONICA SINICA,2023,51(04):1050-1067. DOI： 10.12263/DZXB.20211483.

摘要

盲图像去模糊旨在模糊核未知的情况下从模糊图像恢复清晰图像，这是一个欠定逆问题，需要引入图像先验信息限定解空间.受到SelfDeblur的启发，本文提出了一种基于深度先验的盲图像去模糊算法，结合深度网络与正则化模型对清晰图像与模糊核联合建模，交替迭代估计清晰图像与模糊核.在图像估计子问题中，模糊核参与RGB三通道损失函数的约束下，利用隐含图像平滑性约束的深度卷积神经网络DIP-Net生成清晰图像；在模糊核估计子问题中，直接求取模糊核正则化约束模型的全局极小解，不同于SelfDeblur的全连接网络使用梯度下降法更新模糊核.本文算法结合深度网络实现正则化方法，与监督学习相比，无需成对的模糊/清晰图像数据集训练网络；与传统模型方法相比，无需通过多级金字塔的方式由粗到细地估计模糊核.在模拟与真实模糊图像上的实验结果表明；本文算法能够快速、准确地估计出清晰图像和模糊核，并能够有效抑制图像复原过程中存在的噪声放大问题.

Abstract

Blind image deblurring is the process of removing blurring artifacts from the observation when the blur kernel is unknown

which is a seriously ill-posed problem. It is indispensable to impose prior information constraints on the feasible set. Inspired by the SelfDeblur

in this paper we propose a deep prior-based blind image deblurring method

which uses the deep network and the regularized optimization model to jointly optimize and alternately update the latent image and the blur kernel. Conditioned by the loss of the sum of RGB three-channel errors with the presence of blur kernel

the latent image is estimated using the deep convolutional neural network DIP-Net implicitly involving the smoothness regularizer of the image. The blur kernel estimation subproblem admits the global optimal solution

which is different from the SelfDeblur that applies the fully connected network and takes the gradient descent step to update the blur kernel. Our method uses the structure of the deep network to regularize the latent image. Unlike the supervised image deblurring method

it requires no ground truth of the latent image or the blur kernel. Unlike the traditional model

it requires no progressive transmission from coarse to fine through the multi-level pyramid. Experimental results on simulated and real blur images show that the proposed method achieves a fast and accurate estimation of both the blur kernel and the latent image with efficient noise suppression.

关键词

Keywords

references

NAH S , KIM T H , LEE K M . Deep multi-scale convolutional neural network for dynamic scene deblurring [C ] // Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 257 - 265 .

SCHULER C J , HIRSCH M , HARMELING S , et al . Learning to deblur [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 , 38 ( 7 ): 1439 - 1451 .

YAN R M , SHAO L . Blind image blur estimation via deep learning [J ] . IEEE Transactions on Image Processing , 2016 , 25 ( 4 ): 1910 - 1921 .

TAO X , GAO H Y , SHEN X Y , et al . Scale-recurrent network for deep image deblurring [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 8174 - 8182 .

KUPYN O , BUDZAN V , MYKHAILYCH M , et al . DeblurGAN: Blind motion deblurring using conditional adversarial networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 8183 - 8192 .

KUPYN O , MARTYNIUK T , WU J R , et al . DeblurGAN-v2: Deblurring (orders-of-magnitude) faster and better [C ] // International Conference on Computer Vision . Seoul : IEEE , 2019 : 8877 - 8886 .

ZHANG H G , DAI Y C , LI H D , et al . Deep stacked hierarchical multi-patch network for image deblurring [C ] /// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 5971 - 5979 .

SUIN M , PUROHIT K , RAJAGOPALAN A N . Spatially-attentive patch-hierarchical network for adaptive motion deblurring [C ] // 2020 IEEE/CVF Conference on /Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 3603 - 3612 .

YUAN Y , SU W , MA D D . Efficient dynamic scene deblurring using spatially variant deconvolution network with optical flow guided training [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 3552 - 3561 .

KAUFMAN A , FATTAL R . Deblurring using analysis-synthesis networks pair [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 5810 - 5819 .

LU B Y , CHEN J C , CHELLAPPA R . Unsupervised domain-specific deblurring via disentangled representations [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 10217 - 10226 .

REN D W , ZHANG K , WANG Q L , et al . Neural blind deconvolution using deep priors [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 3338 - 3347 .

ULYNOV D , VEDALDI A , LEMPITSKY V . Deep image prior [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 9446 - 9454 .

CHAN T F , WONG C K . Total variation blind deconvolution [J ] . IEEE Transactions on Image Processing , 1998 , 7 ( 3 ): 370 - 375 .

PERRONE D , FAVARO P . Total variation blind deconvolution: The devil is in the details [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus : IEEE , 2014 : 2909 - 2916 .

XU L , JIA J Y . Two-phase kernel estimation for robust motion deblurring [C ] // European Conference on Computer Vision . Berlin : Springer , 2010 : 157 - 170 .

FERGUS R , SINGH B , HERTZMANN A , et al . Removing camera shake from a single photograph [J ] . ACM Transactions on Graphics , 2006 , 25 ( 3 ): 787 - 794 .

LEVIN A , WEISS Y , DURAND F , et al . Understanding and evaluating blind deconvolution algorithms [C ] // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami : IEEE , 2009 : 1964 - 1971 .

SHAN Q , JIA J Y , AGARWALA A . High-quality motion deblurring from a single image [J ] . ACM Transactions on Graphics , 2008 , 27 ( 3 ): 15 - 19 .

方帅 , 刘远东 , 曹洋 , 等 . 基于模糊结构图的模糊核估计 [J ] . 电子学报 , 2017 , 45 ( 5 ): 1226 - 1233 .

FANG S , LIU Y D , CAO Y , et al . Blur kernel estimation using blurry structure [J ] . Acta Electronica Sinica , 2017 , 45 ( 5 ): 1226 - 1233 . (in Chinese)

余义斌 , 彭念 , 甘俊英 . 凹凸范数比值正则化的快速图像盲去模糊 [J ] . 电子学报 , 2016 , 44 ( 5 ): 1168 - 1173 .

YU Y B , PENG N , GAN J Y . Fast blind image deblurring using ratio of concave norm to convex norm regularization [J ] . Acta Electronica Sinica , 2016 , 44 ( 5 ): 1168 - 1173 . (in Chinese)

MICHAELI T , IRANI M . Blind deblurring using internal patch recurrence [C ] // European Conference on Computer Vision . Zurich : Springer , 2014 : 783 - 798 .

ZHANG H C , YANG J C , ZHANG Y N , et al . Close the loop: Joint blind image restoration and recognition with sparse representation prior [C ] // International Conference on Computer Vision . Barcelona : IEEE , 2011 : 770 - 777 .

徐焕宇 , 孙权森 , 李大禹 , 等 . 基于投影的稀疏表示与非局部正则化图像复原方法 [J ] . 电子学报 , 2014 , 42 ( 7 ): 1299 - 1304 .

XU H Y , SUN Q S , LI D Y , et al . Projection-based image restoration via sparse representation and nonlocal regularization [J ] . Acta Electronica Sinica , 2014 , 42 ( 7 ): 1299 - 1304 . (in Chinese)

REN W Q , CAO X C , PAN J S , et al . Image deblurring via enhanced low-rank prior [J ] . IEEE Transactions on Image Processing , 2016 , 25 ( 7 ): 3426 - 3437 .

PAN J S , SUN D Q , PFISTER H , et al . Deblurring images via dark channel prior [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018 , 40 ( 10 ): 2315 - 2328 .

HE K M , SUN J , TANG X O . Single image haze removal using dark channel prior [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 12 ): 2341 - 2353 .

YAN Y Y , REN W Q , GUO Y F , et al . Image deblurring via extreme channels prior [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 6978 - 6986 .

CHEN L , FANG F M , WANG T T , et al . Blind image deblurring with local maximum gradient prior [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 1742 - 1750 .

ZHANG K , ZUO W M , GU S H , et al . IRCNN: Learning deep CNN denoiser prior for image restoration [C ] // Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 2808 - 2817 .

LI L , PAN J S , LAI W S , et al . Learning a discriminative prior for blind image deblurring [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 6616 - 6625 .

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C ] // International Conference on Medical Image Computing and Computer-Assisted Intervention . Berlin : Springer , 2015 : 234 - 241 .

MATAEV G , MILANFAR P , ELAD M . DeepRED: Deep image prior powered by red [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul : IEEE , 2019 : 1 - 10 .

WANG J Y , CHEN Y B , CHAKRABORTY R , et al . Orthogonal convolutional neural networks [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 11502 - 11512 .

Kingma D P , Ba L J . Adam: A method for stochastic optimization [C ] // International Conference on Learning Representation . San Diego : IEEE , 2015 : 1 - 13 .

LAI W S , HUANG J B , HU Z , et al . A comparative study for single image blind deblurring [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1701 - 1709 .

KOHLER R , HIRSCH M , MOHLER B , et al . Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database [C ] // European Conference on Computer Vision . Berlin : Springer , 2012 : 27 - 40 .

KRISHNAN D , TAY T , FERGUS R . Blind deconvolution using a normalized sparsity measure [C ] // Computer Vision and Pattern Recognition . Colorado Springs : IEEE , 2011 : 233 - 240 .

LEVIN A , WEISS Y , DURAND F , et al . Efficient marginal likelihood optimization in blind deconvolution [C ] // Computer Vision and Pattern Recognition . Providence : IEEE , 2011 : 2657 - 2664 .

CHO S , LEE S . Fast motion deblurring [J ] . ACM Transactions on Graphics , 2009 , 28 ( 5 ): 1 - 8 .

XU L , ZHENG S C , JIA J . Unnatural l0 sparse representation for natural image deblurring [C ] // Computer Vision and Pattern Recognition . Portland : IEEE , 2013 : 1107 - 1114 .

SUN L B , CHO S , WANG J , et al . Edge-based blur kernel estimation using patch priors [C ] // International Conference on Computational Photography . Cambridge : IEEE , 2013 : 1 - 8 .

ZUO W M , REN D W , ZHANG D , et al . Learning iteration-wise generalized shrinkage-thresholding operators for blind deconvolution [J ] . IEEE Transactions on Image Processing , 2016 , 25 ( 4 ): 1751 - 1764 .

常振春 , 禹晶 , 肖创柏 , 等 . 基于稀疏表示和结构自相似性的单幅图像盲解卷积算法 [J ] . 自动化学报 , 2017 , 43 ( 11 ): 1908 - 1919 .

CHANG Z C , YU J , XIAO C B , et al . Single image blind deconvolution using sparse representation and structural self-similarity [J ] . Acta Automatica Sinica , 2017 , 43 ( 11 ): 1908 - 1919 . (in Chinese)

彭天奇 , 禹晶 , 肖创柏 . 基于跨尺度低秩约束的图像盲解卷积算法 [J ] . 自动化学报 , 2022 , 48 ( 10 ): 2508 - 2525 .

PENG T Q , YU J , XIAO C B . Blind image deconvolution via cross-scale low rank prior [J ] . Acta Automatica Sinica , 2022 , 48 ( 10 ): 2508 - 2525 . (in Chinese)

肖创柏 , 王晓宁 , 郭乐宁 , 等 . 一种基于深度先验的盲图像去模糊方法 : CN202210052867.6 [P ] . 2022-04-29 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Multi-Granularity Neural Network Pruning Method with Regularization Mechanism

Projection-Based Image Restoration via Sparse Representation and Nonlocal Regularization

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

Shared Super-Resolution Dual-Branch Network for Spatiotemporal Fusion of Remote Sensing Images

Related Author

BAI Yong-qiang

YU Jing

LI Yi-nong

XIAO Chuang-bai

LIU Qi

CHEN Ying

XU Huan-yu

SUN Quan-sen

Related Institution

Faculty of Information Technology， Beijing University of Technology

The Key Laboratory of Advanced Process Control for Light Industry （Ministry of Education）， Jiangnan University

State Key Laboratory of Applied Optics, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

School of Computer Science and Technology, Nanjing University of Science and Technology

State Key Laboratory of Applied Optics Changchun Institute of Optics Fine Mechanics and Physics Chinese Academy of Sciences Changchun Jilin China

⁰