图像超分辨率重建研究综述

doi:10.3969/j.issn.0372-2112.2020.07.022

摘要
图/表
参考文献(0)
相关文章 (15)

全文: PDF (1622 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要图像超分辨率重建（Super-resolution Reconstruction，SR）是由一张或多张低分辨率图像得到高分辨率图像的过程.近年来，SR技术不断发展，在许多领域被广泛应用.本文在回顾SR技术发展历史的基础上，全面综述了SR技术在各个时期的代表性方法，重点介绍了基于深度学习的图像超分辨率工作.我们从模型类型、网络结构、信息传递方式等方面对各种算法进行了详细评述，并对比了其优缺点.最后探讨了图像超分辨率技术未来的发展方向.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	唐艳秋
	潘泓
	朱亚平
	李新德

关键词 ：图像超分辨率, 深度学习, 图像处理, 方法综述

Abstract：Image super-resolution reconstruction (SR) aims to obtain high-resolution images from one or more low-resolution images.Recently,SR has been developing and widely applied in different fields.This survey retrospects the history of SR technique and provides a comprehensive overview of representative SR methods,with an emphasis on recent deep learning-based approaches.We elaborate the details of various deep learning-based SR methods,including their strengths and weakness,in terms of the deep learning model,architecture,and message pass.Finally,we discuss the possible research directions on SR technique.

Key words： image super-resolution reconstruction deep learning image processing methods review

收稿日期: 2019-05-13 出版日期: 2019-12-26

ZTFLH:

TP391

基金资助:国家自然科学基金（No.61671151，No.61573097，No.91748106）；江苏省自然科学基金（No.BK20181265）；流程工业综合自动化国家重点实验室基金（No.PAL-N201704）；中国传媒大学优秀博导组项目（No.CUC2019A009）；中国传媒大学重大攻关培育项目-媒介事件中的AI新闻生产系统与关键技术（No.CUC19ZD003）

通讯作者: 潘泓 E-mail: enhpan@seu.edu.cn

作者简介: 唐艳秋 女,1995年生,四川达州人.东南大学自动化学院控制科学与工程专业硕士研究生,研究方向为机器学习、数字图像处理等.E-mail:220171510@seu.edu.cn;朱亚平 女,1977年生.博士,教授,主要研究方向为机器学习、深度学习、图像处理、模式识别.E-mail:zhuyaping@cuc.edu.cn;李新德 男,1975年生.博士,教授,主要研究方向为机器感知与学习、数据挖掘、机器视觉、智能信息融合和不确定推理、智能机器人.E-mail:xindeli@seu.edu.cn

引用本文:

唐艳秋, 潘泓, 朱亚平, 李新德. 图像超分辨率重建研究综述[J]. 电子学报, 2020, 48(7): 1407-1420. TANG Yan-qiu, PAN Hong, ZHU Ya-ping, LI Xin-de. A Survey of Image Super-Resolution Reconstruction. Acta Electronica Sinica, 2020, 48(7): 1407-1420.

链接本文:

http://www.ejournal.org.cn/CN/10.3969/j.issn.0372-2112.2020.07.022 或 http://www.ejournal.org.cn/CN/Y2020/V48/I7/1407

[1] Tsai R Y.Multiframe image restoration and registration[J].Advance Computer Visual and Image Processing,1984,1:317-339.
[2] 吴秀秀,肖珊,张煜.基于配准的肺4D-CT图像超分辨率重建[J].电子学报,2015,43(2):383-386. WU Xiu-xiu,XIAO Shan,ZHANG Yu.Registration based super-resolution reconstruction for lung 4D-CT image[J].Acta Electronica Sinica,2015,43(2):383-386.(in Chinese)
[3] Jia Z,Wang H,Xiong Z,et al.Fast face hallucination with sparse representation for video surveillance[A].First Asian Conference on Pattern Recognition (ACPR)[C].USA:IEEE,2011.179-183.
[4] Hu M G,Wang J F,Ge Y.Super-resolution reconstruction of remote sensing images using multifractal analysis[J].Sensors,2009,9(11):8669-8683.
[5] Harris J L.Difiraction and resolving power[J].Journal of the Optical Society of America,1964,54(7):931-936.
[6] Goodman J W,Cox M E.Introduction to Fourier Optics[M].McGraw-Hill,1968.
[7] Tsai R Y,Huang T S.Multiframe image restoration and registration[J].Advances in Computer Vision and Image Processing,1984,1:317-339.
[8] Borman S,Stevenson R.Spatial Resolution Enhancement of Low-resolution Image Sequences:A Comprehensive Review with Directions for Future Research,Technical Report[R].Laboratory Image and Signal Analysis,University of Notre Dame,1998.
[9] Park S C,Min K P,Kang M G.Super-resolution image reconstruction:a technical overview[J].Signal Processing Magazine IEEE,2003,20(3):21-36.
[10] 苏衡,周杰,张志浩.超分辨率图像重建方法综述[J].自动化学报,2013,39(8):202-1213.
[11] Huang D,Liu H.A short survey of image super resolution algorithms[J].Journal of Computer Science Technology Updates,2015,2(2):19-29.
[12] Hayat K.Super-resolution via deep learning[J].arXiv Peprint,2017,arXiv:1706.09077.
[13] Sun X,Xiao-Guang L I,Jia-Feng L I,et al.Review on deep learning based image super-resolution restoration algorithms[J].Acta Automatica Sinica,2017,43(5):697-709.
[14] Yang W,Zhang X,Tian Y,et al.Deep learning for single image super-resolution:A brief review[J].arXiv Preprint,2018,arXiv:1808.03344.
[15] Kim S P,Bose N K,Valenzuela H M.Recursive reconstruction of high resolution image from noisy under sampled multiframes[J].IEEE Transactions on Acoustics Speech & Signal Processing,1990,38(6):1013-1027.
[16] Rhee S,Kang M G.Discrete cosine transform based regularized high-resolution image reconstruction algorithm[J].Optical Engineering,1999,38(8):1348-1356.
[17] 王相海,张晓阳,毕晓昀,陶兢喆.小波域多角度轮廓模板变分模型的单幅图像超分辨率重建[J].电子学报,2018,46(9):2256-2262. WANG Xiang-hai,ZHAO Xiao-yang,BI Xiao-yun,TAO Jing-zhe.Single image super-resolution reconstruction approach based on multi-angle contour templates variational calculus model in wavelet domain[J].Acta Electronica Sinica,2018,46(9):2256-2262.(in Chinese)
[18] Nguyen N,Milanfar P.An efficient wavelet-based algorithm for image super-resolution[A].Proceedings of the 2000 International Conference on Image Processing[C].USA:IEEE,2000.351-354.
[19] Panagiotopoulou A,Anastassopoulos V.Super-resolution image reconstruction employing Kriging interpolation technique[A].The 14th International Workshop on Systems,Signals and Image Processing and 6th EURASIP Conference focused on Speech and Image Processing,Multimedia Communications and Services[C].USA:IEEE,2007.144-147.
[20] Irani M,Peleg S.Super resolution from image sequences[A].Proceedings of the 10th International Conference on Pattern Recognition[C].USA:IEEE,1990.115-120.
[21] Schultz R R,Stevenson R L.An Bayesian approach to image expansion for improved definition[J].IEEE Transactions on Image Processing,1994,3(3):233-242.
[22] Schultz R R,Stevenson R L.Improved definition video frame enhancement[A].International Conference on Acoustics,Speech,and Signal Processing(ICASSP-95)[C].USA:IEEE,1995.2169-2172.
[23] Stark H,Oskoui P.High-resolution image recovery from image-plane arrays,using convex projections[J].Journal of the Optical Society of America A Optics & Image Science,1989,6(11):1715.
[24] Elad M,Feuer A.Restoration of a single super-resolution image from several blurred,noisy,and under sampled measured images[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,1997,6(12):1646-58.
[25] Freeman W T,Jones T R,Pasztor E C.Example-based super-resolution[J].IEEE Computer Graphics & Applications,2002,22(2):56-65.
[26] Glasner D,Bagon S,Irani M.Super-resolution from a single image[A].International Conference on Computer Vision[C].USA:IEEE,2009.349-356.
[27] Chang H,Yeung D Y,Xiong Y.Super-resolution through neighbor embedding[A].Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR 2004)[C].USA:IEEE,2004.I-I.
[28] Yang J,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE Trans on Image Process,2010,19(11):2861-2873.
[29] 詹曙,方琪,杨福猛,常乐乐,闫婷.基于耦合特征空间下改进字典学习的图像超分辨率重建[J].电子学报,2016,44(5):1189-1195. ZHAN Shu,FANG Qi,YANG Fu-meng,CHANG Le-le,YAN Ting.Image super-resolution reconstruction via improved dictionary learning based on coupled feature space[J].Acta Electronica Sinica,2016,44(5):1189-1195.(in Chinese)
[30] Dong C,Loy C C,He K,et al.Learning a deep convolutional network for image super-resolution[A].European Conference on Computer Vision[C].Springer,Cham,2014.184-199.
[31] Tai Y,Yang J,Liu X,et al.MemNet:A persistent memory network for image restoration[A].IEEE International Conference on Computer Vision[C].USA:IEEE,2017.4549-4557.
[32] Zhang Y,Tian Y,Kong Y,et al.Residual dense network for image super-resolution[A].The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].USA:IEEE,2018.DOI:10.1109/CVPR.2018.00262.
[33] Shi W,Caballero J,Huszár,Ferenc,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[A].2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR)[C].USA:IEEE,2016.DOI:10.1109/CVPR.2016.207.
[34] Dong C,Loy C C,Tang X.Accelerating the super-resolution convolutional neural network[A].European Conference on Computer Vision[C].Springer,Cham,2016.391-407.
[35] Liang Y,Wang J,Zhou S,et al.Incorporating image priors with deep convolutional neural networks for image super-resolution[J].Neurocomputing,2016,194:340-347.
[36] Wang Z,Liu D,Yang J,et al.Deep networks for image super-resolution with sparse prior[A].Proceedings of the IEEE International Conference on Computer Vision[C].USA:IEEE,2015.370-378.
[37] Yang W,Feng J,Yang J,et al.Deep edge guided recurrent residual learning for image super-resolution[J].IEEE Transactions on Image Processing,2017,26(12):5895-5907.
[38] Kim J,Kwon Lee J,Mu Lee K.Accurate image super-resolution using very deep convolutional networks[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.1646-1654.
[39] Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[A].Computer Vision and Pattern Recognition[C].USA:IEEE,2016.1637-1645.
[40] Tai Y,Yang J,Liu X.Image super-resolution via deep recursive residual network[A].IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE Computer Society,2017.2790-2798.
[41] Zhang Y,Li K,Li K,et al.Image super-resolution using very deep residual channel attention networks[A].Proceedings of the European Conference on Computer Vision (ECCV)[C].USA:IEEE,2018.286-301.
[42] Liao Q,Poggio T.Bridging the gaps between residual learning,recurrent neural networks and visual cortex[J].arXiv Preprint,2016,arXiv:1604.03640.
[43] Han W,Chang S,Liu D,et al.Image super-resolution via dual-state recurrent networks[J].arXiv Preprint,2018,arXiv:1805.02704.
[44] Li J,Fang F,Mei K,et al.Multi-scale residual network for image super-resolution[A].Proceedings of the European Conference on Computer Vision (ECCV)[C].USA:IEEE,2018.517-532.
[45] Tong T,Li G,Liu X,et al.Image super-resolution using dense skip connections[A].IEEE International Conference on Computer Vision[C].USA:IEEE Computer Society,2017.4809-4817.
[46] Lai W S,Huang J B,Ahuja N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[A].IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE Computer Society,2017.5835-5843.
[47] Lim B,Son S,Kim H,et al.Enhanced deep residual networks for single image super-resolution[A].Computer Vision and Pattern Recognition Workshops[C].USA:IEEE,2017.1132-1140.
[48] Huang G,Liu Z,Van Der Maaten L,et al.Densely connected convolutional networks[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2017.4700-4708.
[49] Ledig C,Theis L,Huszar F,et al.Photo-realistic single image super-resolution using a generative adversarial network[J].arXiv Preprint,2016,arXiv:1609.04802.
[50] Johnson J,Alahi A,Fei-Fei L.Perceptual losses for real-time style transfer and super-resolution[A].European Conference on Computer Vision[C].Springer,Cham,2016.694-711.
[51] Wang Y,Perazzi F,McWilliams B,et al.A fully progressive approach to single-image super-resolution[J].arXiv Preprint,2018,arXiv:1804.02900.
[52] Wang X,Yu K,Dong C,et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.606-615.
[53] Chen Y,Tai Y,Liu X,et al.FSRNet:End-to-end learning face super-resolution with facial priors[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.2492-2501.
[54] Zhang K,Zuo W,Zhang L.Learning a single convolutional super-resolution network for multiple degradations[A].IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2018.DOI:10.1109/CVPR.2018.00344.
[55] Bulat A,Yang J,Tzimiropoulos G.To learn image super-resolution,use a GAN to learn how to do image degradation first[A].Proceedings of the European Conference on Computer Vision (ECCV)[C].USA:IEEE,2018.185-200.
[56] Sønderby C K,Caballero J,Theis L,et al.Amortised map inference for image super-resolution[J].arXiv Preprint,2016,arXiv:1610.04490.
[57] Gao J,Guo Y,Yin M.Restricted Boltzmann machine approach to couple dictionary training for image super-resolution[A].IEEE International Conference on Image Processing[C].USA:IEEE,2014.499-503.
[58] Zhou Y,Qu Y,Xie Y,et al.Image super-resolution using deep belief networks[A].International Conference on Internet Multimedia Computing and Service[C].USA:ACM,2014.28.
[59] Cui Z,Chang H,Shan S,et al.Deep network cascade for image super-resolution[A].Computer Vision-ECCV2014[M].Springer International Publishing,2014.49-64.
[60] Mao X J,Shen C,Yang Y B.Image restoration using convolutional auto-encoders with symmetric skip connections[J].arXiv Preprint,2016,arXiv:1606.08921.
[61] Hu X,Mu H,Zhang X,et al.Meta-SR:A magnification-arbitrary network for super-resolution[J].arXiv Preprint,2019,arXiv:1903.00875.