A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning

WU Jing; YE Xiao-jing; HUANG Feng; CHEN Li-qiong; WANG Zhi-feng; LIU Wen-xi

doi:10.12263/DZXB.20220091

您当前的位置：

首页 >

文章列表页 >

A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning

SURVEYS AND REVIEWS | 更新时间：2025-12-08

- A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning
- ACTA ELECTRONICA SINICA Vol. 50, Issue 9, Pages: 2265-2294(2022)
- 作者机构：
  
  1.福州大学机械工程及自动化学院，福建福州 350116
  2.福州大学先进技术创新研究院，福建福州 350116
  3.福州大学计算机与大数据学院，福建福州 350116
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20220091
  CLC： TP751;TP391
- Received：17 January 2022，
  
  Revised：2022-05-13，
  
  Published：25 September 2022
- 稿件说明：
移动端阅览
吴靖,叶晓晶,黄峰等.基于深度学习的单帧图像超分辨率重建综述[J].电子学报,2022,50(09):2265-2294.

WU Jing,YE Xiao-jing,HUANG Feng,et al.A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(09):2265-2294.
吴靖,叶晓晶,黄峰等.基于深度学习的单帧图像超分辨率重建综述[J].电子学报,2022,50(09):2265-2294. DOI： 10.12263/DZXB.20220091.

WU Jing,YE Xiao-jing,HUANG Feng,et al.A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(09):2265-2294. DOI： 10.12263/DZXB.20220091.

摘要

图像超分辨率重建是计算机视觉中的基本图像处理技术之一，不仅可以提高图像分辨率改善图像质量，还可以辅助其他计算机视觉任务. 近年来，随着人工智能浪潮的兴起，基于深度学习的图像超分辨率重建也取得了显著进展. 本文在简述图像超分辨率重建方法的基础上，全面综述了基于深度学习的单帧图像超分辨率重建的技术架构及研究历程，包括数据集构建方式、网络模型基本框架以及用于图像质量评估的主、客观评价指标，重点介绍了根据网络结构及图像重建效果划分的基于卷积神经网络的方法、基于生成对抗网络的方法以及基于Transformer的方法，并对相关网络模型加以评述和对比，最后依据网络模型和超分辨率重建挑战赛相关内容，展望了图像超分辨率重建未来的发展趋势.

Abstract

Image super-resolution reconstruction is one of the basic image processing techniques in computer vision

which can not only improve image resolution and image quality

but also assist other computer vision tasks. In recent years

with the rise of artificial intelligence

deep-learning-based image super-resolution reconstruction has also made remarkable progress. Based on a brief description of the image super-resolution reconstruction methodology

this paper comprehensively reviews the technical architecture and research process of deep-learning-based single image super-resolution reconstruction

including the method of datasets construction

the basic framework of the network model

the subjective and objective evaluation metrics for image quality evaluation. The methods based on convolutional neural networks

generative adversarial networks and Transformer

which are divided according to network structure and image reconstruction effect are mainly introduced

and related network models are reviewed and compared. Finally

the future development trend of image super-resolution reconstruction is prospected according to the related content of network model and super-resolution reconstruction challenges.

关键词

Keywords

references

DAI D X , WANG Y J , CHEN Y H , et al . Is image super-resolution helpful for other vision tasks? [C]// 2016 IEEE Winter Conference on Applications of Computer Vision . Lake Placid : IEEE , 2016 : 1 - 9 .

BAI Y C , ZHANG Y Q , DING M L , et al . SOD-MTGAN: Amall object detection via multi-task generative adversarial network [C]// Proceedings of the European Conference on Computer Vision(ECCV 2018) . Munich : Springer , 2018 : 210 - 226 .

BEI Y J , DAMIAN A , HU S J , et al . New techniques for preserving global structure and denoising with low information loss in single-image super-resolution [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Salt Lake City : IEEE , 2018 : 987 - 9877 .

吴秀秀 , 肖珊 , 张煜 . 基于配准的肺4D-CT图像超分辨率重建 [J]. 电子学报 , 2015 , 43 ( 2 ): 383 - 386 .

WU X X , XIAO S , ZHANG Y . Registration based super-resolution reconstruction for lung 4D-CT image [J]. Acta Electronica Sinica , 2015 , 43 ( 2 ): 383 - 386 . (in Chinese)

YIN Y , ROBINSON J , ZHANG Y L , et al . Joint super-resolution and alignment of tiny faces [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 12693 - 12700 .

HUANG Y W , ZHENG F , WANG D Y , et al . Super-resolution and inpainting with degraded and upgraded generative adversarial networks [C]// Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence . Yokohama : ACM , 2021 : 645 - 651 .

WANG Z Y , JIANG K , YI P , et al . Ultra-dense GAN for satellite imagery super-resolution [J]. Neurocomputing , 2020 , 398 : 328 - 337 .

GAO G W , YU Y , XIE J , et al . Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution [J]. Pattern Recognition , 2021 , 110 : 107539 .

李云红 , 王珍 , 张凯兵 , 等 . 基于学习的图像超分辨重建方法综述 [J]. 计算机工程与应用 , 2018 , 54 ( 15 ): 13 - 21 .

LI Y H , WANG Z , ZHANG K B , et al . Survey on example learning-based single image super-resolution technique [J]. Computer Engineering and Applications , 2018 , 54 ( 15 ): 13 - 21 . (in Chinese)

孙旭 , 李晓光 , 李嘉锋 , 等 . 基于深度学习的图像超分辨率复原研究进展 [J]. 自动化学报 , 2017 , 43 ( 5 ): 697 - 709 .

SUN X , LI X G , LI J F , et al . Review on deep learning based image super-resolution restoration algorithms [J]. Acta Automatica Sinica , 2017 , 43 ( 5 ): 697 - 709 . (in Chinese)

刘颖 , 朱丽 , 林庆帆 , 等 . 图像超分辨率技术的回顾与展望 [J]. 计算机科学与探索 , 2020 , 14 ( 2 ): 181 - 199 .

LIU Ying , ZHU Li , LIM K P , et al . Review and prospect of image super-resolution technology [J]. Journal of Frontiers of Computer Science and Technology , 2020 , 14 ( 2 ): 181 - 199 . (in Chinese)

HARRIS J L . Diffraction and resolving power [J]. Journal of the Optical Society of America , 1964 , 54 ( 7 ): 931 - 936 .

GOODMAN J W , COX M E . Introduction to fourier optics [J]. Physics Today , 1969 , 22 ( 4 ): 97 - 101 .

TSAI R Y , HUANG T S . Multiframe image restoration and registration [J]. Advances in Computer Vision and Image Processing , 1984 , 1 ( 2 ): 317 - 339 .

DUCHON C E . Lanczos filtering in one and two dimensions [J]. Journal of Applied Meteorology , 1979 , 18 ( 8 ): 1016 - 1022 .

KEYS R . Cubic convolution interpolation for digital image processing [J]. IEEE Transactions on Acoustics, Speech, and Signal Processing , 1981 , 29 ( 6 ): 1153 - 1160 .

STARK H , OSKOUI P . High-resolution image recovery from image-plane arrays, using convex projections [J]. Journal of the Optical Society of America . A, Optics and Image Science, 1989 , 6 ( 11 ): 1715 - 1726 .

IRANI M , PELEG S . Improving resolution by image registration [J]. CVGIP: Graphical Models and Image Processing , 1991 , 53 ( 3 ): 231 - 239 .

SCHULTZ R R , STEVENSON R L . A Bayesian approach to image expansion for improved definition [J]. IEEE Transactions on Image Processing , 1994 , 3 ( 3 ): 233 - 242 .

伍政华 , 孙明健 , 顾宗山 , 等 . 基于二阶广义方向性全变分的图像超分辨率重建方法 [J]. 电子学报 , 2017 , 45 ( 11 ): 2625 - 2632 .

WU Z H , SUN M J , GU Z S , et al . Second-order directional total generalized variation regularization for image super-resolution [J]. Acta Electronica Sinica , 2017 , 45 ( 11 ): 2625 - 2632 . (in Chinese)

王相海 , 赵晓阳 , 毕晓昀 , 等 . 小波域多角度轮廓模板变分模型的单幅图像超分辨率重建 [J]. 电子学报 , 2018 , 46 ( 9 ): 2256 - 2262 .

WANG X H , ZHAO X Y , BI X Y , et al . Single image super-resolution reconstruction approach based on multi-angle contour templates variational calculus model in wavelet domain [J]. Acta Electronica Sinica , 2018 , 46 ( 9 ): 2256 - 2262 . (in Chinese)

FREEMAN W T , PASZTOR E C , CARMICHAEL O T . Learning low-level vision [J]. International Journal of Computer Vision , 2000 , 40 : 25 - 47 .

CHANG H , YEUNG D Y , XIONG Y M . Super-resolution through neighbor embedding [C]// Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Washington : IEEE , 2004 : 275 - 282 .

YANG J C , WRIGHT J , HUANG T , et al . Image super-resolution as sparse representation of raw image patches [C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition . Anchorage : IEEE , 2008 : 1 - 8 .

TIMOFTE R , DE V , GOOL L V . Anchored neighborhood regression for fast example-based super-resolution [C]// 2013 IEEE International Conference on Computer Vision . Sydney : IEEE , 2013 : 1920 - 1927 .

潘宗序 , 禹晶 , 肖创柏 , 等 . 基于自适应多字典学习的单幅图像超分辨率算法 [J]. 电子学报 , 2015 , 43 ( 2 ): 209 - 216 .

PAN Z X , YU J , XIAO C B , et al . Single image super resolution based on adaptive multi-dictionary learning [J]. Acta Electronica Sinica , 2015 , 43 ( 2 ): 209 - 216 . (in Chinese)

PARK S C , PARK M K , KANG M G . Super-resolution image reconstruction: A technical overview [J]. IEEE Signal Processing Magazine , 2003 , 20 ( 3 ): 21 - 36 .

江静 , 张雪松 . 图像超分辨率重建算法综述 [J]. 红外技术 , 2012 , 34 ( 1 ): 24 - 30 .

JIANG J , ZHANG X S . A review of super-resolution reconstruction algorithms [J]. Infrared Technology , 2012 , 34 ( 1 ): 24 - 30 . (in Chinese)

苏衡 , 周杰 , 张志浩 . 超分辨率图像重建方法综述 [J]. 自动化学报 , 2013 , 39 ( 8 ): 1202 - 1213 .

SU H , ZHOU J , ZHANG Z H . Survey of super-resolution image reconstruction methods [J]. Acta Automatica Sinica , 2013 , 39 ( 8 ): 1202 - 1213 . (in Chinese)

YANG W M , ZHANG X C , TIAN Y P , et al . Deep learning for single image super-resolution: A brief review [J]. IEEE Transactions on Multimedia , 2019 , 21 ( 12 ): 3106 - 3121 .

史振威 , 雷森 . 图像超分辨重建算法综述 [J]. 数据采集与处理 , 2020 , 35 ( 1 ): 1 - 20 .

SHI Z W , LEI S . Review of image super-resolution reconstruction [J]. Journal of Data Acquisition and Processing , 2020 , 35 ( 1 ): 1 - 20 . (in Chinese)

张德 , 林青宇 , 郭茂祖 . 单幅图像超分辨重建的深度学习方法综述 [J]. 计算机工程与应用 , 2021 , 57 ( 22 ): 28 - 41 .

ZHANG D , LIN Q Y , GUO M Z . Review of single image super-resolution based on deep learning [J]. Computer Engineering and Applications , 2021 , 57 ( 22 ): 28 - 41 . (in Chinese)

黄健 , 赵元元 , 郭苹 , 等 . 深度学习的单幅图像超分辨率重建方法综述 [J]. 计算机工程与应用 , 2021 , 57 ( 18 ): 13 - 23 .

HUANG J , ZHAO Y Y , GUO P , et al . Survey of single image super-resolution based on deep learning [J]. Computer Engineering and Applications , 2021 , 57 ( 18 ): 13 - 23 . (in Chinese)

YANG Z Y , SHI P , PAN D . A survey of super-resolution based on deep learning [C]// 2020 International Conference on Culture-oriented Science & Technology(ICCST) . Beijing : IEEE , 2020 : 514 - 518 .

李佳星 , 赵勇先 , 王京华 . 基于深度学习的单幅图像超分辨率重建算法综述 [J]. 自动化学报 , 2021 , 47 ( 10 ): 2341 - 2363 .

LI J X , ZHAO Y X , WANG J H . A review of single image super-resolution reconstruction algorithms based on deep learning [J]. Acta Automatica Sinica , 2021 , 47 ( 10 ): 2341 - 2363 . (in Chinese)

唐艳秋 , 潘泓 , 朱亚平 , 等 . 图像超分辨率重建研究综述 [J]. 电子学报 , 2020 , 48 ( 7 ): 1407 - 1420 .

TANG Y Q , PAN H , ZHU Y P , et al . A survey of image super-resolution reconstruction [J]. Acta Electronica Sinica , 2020 , 48 ( 7 ): 1407 - 1420 . (in Chinese)

WANG Z H , CHEN J , HOI S C H . Deep learning for image super-resolution: A survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 10 ): 3365 - 3387 .

ANWAR S , KHAN S , BARNES N . A deep journey into super-resolution [J]. ACM Computing Surveys , 2021 , 53 ( 3 ): 1 - 34 .

CHEN H G , HE X H , QING L B , et al . Real-world single image super-resolution: A brief review [J]. Information Fusion , 2022 , 79 : 124 - 145 .

AGUSTSSON E , TIMOFTE R . NTIRE 2017 challenge on single image super-resolution: Dataset and study [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu : IEEE , 2017 : 1122 - 1131 .

GU S H , LUGMAYR A , DANELLJAN M , et al . DIV8K: DIVerse 8K resolution image dataset [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW) . Seoul : IEEE , 2019 : 3512 - 3516 .

CAI J R , ZENG H , YONG H W , et al . Toward real-world single image super-resolution: A new benchmark and a new model [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Seoul : IEEE , 2019 : 3086 - 3095 .

WEI P X , XIE Z W , LU H N , et al . Component divide-and-conquer for real-world image super-resolution [C]// Computer Vision-ECCV 2020 . Glasgow : Springer , 2020 : 101 - 117 .

ZHANG K , ZUO W M , ZHANG L . Learning a single convolutional super-resolution network for multiple degradations [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 3262 - 3271 .

GU J J , LU H N , ZUO W M , et al . Blind super-resolution with iterative kernel correction [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 1604 - 1613 .

LUO Z X , HUANG Y , LI S , et al . Unfolding the alternating optimization for blind super resolution [EB/OL]. ( 2020-11-25 )[ 2022-01-16 ]. https://arxiv.org/abs/2010.02631 https://arxiv.org/abs/2010.02631 .

ZHANG K , LIANG J Y , VAN GOOL L , et al . Designing a practical degradation model for deep blind image super-resolution [C]// 2021 IEEE/CVF International Conference on Computer Vision(ICCV) . Montreal : IEEE , 2021 : 4771 - 4780 .

WANG X T , XIE L B , DONG C , et al . Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data [EB/OL]. ( 2021-08-17 )[ 2022-01-16 ]. https://arxiv.org/abs/2107.10833 https://arxiv.org/abs/2107.10833 .

BELL-KLIGLER S , SHOCHER A , IRANI M . Blind super-resolution kernel estimation using an Internal-GAN [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems(NeurIPS) . Vancouver : Curran Associates, Inc. , 2019 : 284 - 293 .

FRITSCHE M , GU S H , TIMOFTE R . Frequency separation for real-world super-resolution [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW) . Seoul : IEEE , 2019 : 3599 - 3608 .

JI X Z , TAO G P , CAO Y , et al . Frequency consistent adaptation for real world super resolution [EB/OL]. ( 2020-12-18 )[ 2022-01-16 ]. https://arxiv.org/abs/2012.10102 https://arxiv.org/abs/2012.10102 .

YANG J C , WRIGHT J , HUANG T S , et al . Image super-resolution via sparse representation [J]. IEEE Transactions on Image Processing , 2010 , 19 ( 11 ): 2861 - 2873 .

TIMOFTE R , DE SMET V , VAN GOOL L . A+: Adjusted anchored neighborhood regression for fast super-resolution [C]// Computer Vision-ACCV 2014 . Singapore : Springer , 2015 : 111 - 126 .

SCHULTER S , LEISTNER C , BISCHOF H . Fast and accurate image upscaling with super-resolution forests [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston : IEEE , 2015 : 3791 - 3799 .

KIM J , LEE J K , LEE K M . Accurate image super-resolution using very deep convolutional networks [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1646 - 1654 .

DONG C , LOY C C , TANG X O . Accelerating the super-resolution convolutional neural network [C]// Computer Vision-ECCV 2016 . Amsterdam : Springer , 2016 : 391 - 407 .

LIN T Y , MAIRE M , BELONGIE S , et al . Microsoft COCO: Common objects in context [C]// Computer Vision-ECCV 2014 . Zurich : Springer , 2014 : 740 - 755 .

DENG J , DONG W , SOCHER R , et al . ImageNet: A large-scale hierarchical image database [C]// 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami : IEEE , 2009 : 248 - 255 .

TIMOFTE R , AGUSTSSON E , Van GOOL L , et al . NTIRE 2017 challenge on single image super-resolution: Methods and results [C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Honolulu : IEEE , 2017 : 1110 - 1121 .

JI X Z , CAO Y , TAI Y , et al . Real-world super-resolution via kernel estimation and noise injection [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Seattle : IEEE , 2020 : 1914 - 1923 .

IGNATOV A , KOBYSHEV N , TIMOFTE R , et al . DSLR-quality photos on mobile devices with deep convolutional networks [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 3297 - 3305 .

WANG X T , YU K , DONG C , et al . Recovering realistic texture in image super-resolution by deep spatial feature transform [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 606 - 615 .

BEVILACQUA M , ROUMY A , GUILLEMOT C , et al . Low-complexity single-image super-resolution based on nonnegative neighbor embedding [C]// Proceedings of the British Machine Vision Conference 2012 . Guildford : BMVA Press , 2012 : 135.1- 135 . 10

ZEYDE R , ELAD M , PROTTER M . On single image scale-up using sparse-representations [C]// Proceedings of the 7th International Conference on Curves and Surfaces . Avignon : Springer , 2012 : 711 - 730 .

MARTIN D , FOWLKES C , TAL D , et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings Eighth IEEE International Conference on Computer Vision(ICCV) . Vancouver : IEEE , 2001 : 416 - 423 .

HUANG J B , SINGH A , AHUJA N . Single image super-resolution from transformed self-exemplars [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston : IEEE , 2015 : 5197 - 5206 .

FUJIMOTO A , OGAWA T , YAMAMOTO K , et al . Manga109 dataset and creation of metadata [C]// Proceedings of the 1st International Workshop on coMics ANalysis, Processing and Understanding . Cancun : ACM , 2016 : (2) 1 - 5 .

BLAU Y , MECHREZ R , TIMOFTE R , et al . The 2018 PIRM challenge on perceptual image super-resolution [C]// Lecture Notes in Computer Science . Munich : Springer , 2019 : 334 - 355 .

DONG C , LOY C C , HE K M , et al . Learning a deep convolutional network for image super-resolution [C]// Computer Vision-ECCV 2014 . Zurich : Springer , 2014 : 184 - 199 .

DONG C , LOY C C , HE K M , et al . Image super-resolution using deep convolutional networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 , 38 ( 2 ): 295 - 307 .

KIM J , LEE J K , LEE K M . Deeply-recursive convolutional network for image super-resolution [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1637 - 1645 .

HU X C , MU H Y , ZHANG X Y , et al . Meta-SR: A magnification-arbitrary network for super-resolution [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 1575 - 1584 .

SHI W Z , CABALLERO J , HUSZÁR F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1874 - 1883 .

FAN Y C , SHI H H , YU J H , et al . Balanced two-stage residual networks for image super-resolution [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu : IEEE , 2017 : 1157 - 1164 .

ZHANG Y , LI K , LI K , et al . Residual non-local attention networks for image restoration [C]// Proceedings of the Seventh International Conference on Learning Representations(ICLR 2019) . New Orleans : ICLR Press , 2019 : 1 - 18 .

LAI W S , HUANG J B , AHUJA N , et al . Deep laplacian pyramid networks for fast and accurate super-resolution [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 5835 - 5843 .

LAI W S , HUANG J B , AHUJA N , et al . Fast and accurate image super-resolution with deep Laplacian pyramid networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 11 ): 2599 - 2613 .

CHUDASAMA V , UPLA K . E-ProSRNet: An enhanced progressive single image super-resolution approach [J]. Computer Vision and Image Understanding , 2020 , 200 : 103038 .

HARIS M , SHAKHNAROVICH G , UKITA N . Deep back-projection networks for super-resolution [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 1664 - 1673 .

HARIS M , SHAKHNAROVICH G , UKITA N . Deep back-Projection Networks for single image super-resolution [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 12 ): 4323 - 4337 .

HAN W , CHANG S Y , LIU D , et al . Image super-resolution via dual-state recurrent networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 1654 - 1663 .

LI Z , YANG J L , LIU Z , et al . Feedback network for image super-resolution [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 3862 - 3871 .

LUGMAYR A , DANELLJAN M , TIMOFTE R , et al . NTIRE 2020 challenge on real-world image super-resolution: Methods and results [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Seattle : IEEE , 2020 : 2058 - 2076 .

STREIJL R C , WINKLER S , HANDS D S . Mean opinion score(MOS) revisited: Methods and applications, limitations and alternatives [J]. Multimedia Systems , 2016 , 22 ( 2 ): 213 - 227 .

张凯兵 , 朱丹妮 , 王珍 , 等 . 超分辨图像质量评价综述 [J]. 计算机工程与应用 , 2019 , 55 ( 4 ): 31 - 40, 47 .

ZHANG K B , ZHU D N , WANG Z , et al . Survey of super-resolution images quality assessment [J]. Computer Engineering and Applications , 2019 , 55 ( 4 ): 31 - 40, 47 . (in Chinese)

高敏娟 , 党宏社 , 魏立力 , 等 . 全参考图像质量评价回顾与展望 [J]. 电子学报 , 2021 , 49 ( 11 ): 2261 - 2272 .

GAO M J , DANG H S , WEI L L , et al . Review and prospect of full reference image quality assessment [J]. Acta Electronica Sinica , 2021 , 49 ( 11 ): 2261 - 2272 . (in Chinese)

WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment: from error visibility to structural similarity [J]. IEEE Transactions on Image Processing , 2004 , 13 ( 4 ): 600 - 612 .

SHEIKH H R , BOVIK A C , DE VECIANA G . An information fidelity criterion for image quality assessment using natural scene statistics [J]. IEEE Transactions on Image Processing , 2005 , 14 ( 12 ): 2117 - 2128 .

ZHANG R , ISOLA P , EFROS A A , et al . The unreasonable effectiveness of deep features as a perceptual metric [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 586 - 595 .

MITTAL A , SOUNDARARAJAN R , BOVIK A C . Making a “completely blind” image quality analyzer [J]. IEEE Signal Processing Letters , 2013 , 20 ( 3 ): 209 - 212 .

VENKATANATH N , PRANEETH D , BH M C , et al . Blind image quality evaluation using perception based features [C]// 2015 Twenty First National Conference on Communications(NCC) . Mumbai : IEEE , 2015 : 1 - 6 .

MA C , YANG C Y , YANG X K , et al . Learning a no-reference quality metric for single-image super-resolution [J]. Computer Vision and Image Understanding , 2017 , 158 : 1 - 16 .

LEDIG C , THEIS L , HUSZÁR F , et al . Photo-realistic single image super-resolution using a generative adversarial network [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 105 - 114 .

CHEN H T , WANG Y H , GUO T Y , et al . Pre-trained image processing transformer [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 12294 - 12305 .

HUBEL D H , WIESEL T N . Receptive fields, binocular interaction and functional architecture in the cat's visual cortex [J]. The Journal of Physiology , 1962 , 160 ( 1 ): 106 - 154 .

LI Z W , LIU F , YANG W J , et al . A survey of convolutional neural networks: Analysis, applications, and prospects [J]. IEEE Transactions on Neural Networks and Learning Systems , 2021 : 1 - 21 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 770 - 778 .

MAO X , SHEN C , YANG Y . Image restoration using convolutional auto-encoders with symmetric skip connections [C]// Proceedings of the Advances in Neural Information Processing Systems 29(NIPS 2016 )[C]. Barcelona: Curran Associates , Inc. , 2016 .

LIM B , SON S , KIM H , et al . Enhanced deep residual networks for single image super-resolution [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Honolulu : IEEE , 2017 : 1132 - 1140 .

LI J , FANG F , MEI K , et al . Multi-scale residual network for image super-resolution [C]// Proceedings of the European Conference on Computer Vision . Munich : Springer , 2018 : 527 - 542 .

LAN R S , SUN L , LIU Z B , et al . Cascading and enhanced residual networks for accurate single-image super-resolution [J]. IEEE Transactions on Cybernetics , 2021 , 51 ( 1 ): 115 - 125 .

南方哲 , 钱育蓉 , 行艳妮 , 等 . 基于深度学习的单图像超分辨率重建研究综述 [J]. 计算机应用研究 , 2020 , 37 ( 2 ): 321 - 326 .

NAN F Z , QIAN Y R , XING Y N , et al . Survey of single image super resolution based on deep learning [J]. Application Research of Computers , 2020 , 37 ( 2 ): 321 - 326 . (in Chinese)

杨祎玥 , 伏潜 , 万定生 . 基于深度循环神经网络的时间序列预测模型 [J]. 计算机技术与发展 , 2017 , 27 ( 3 ): 35 - 38, 43 .

YANG Y Y , FU Q , WAN D S . A prediction model for time series based on deep recurrent neural network [J]. Computer Technology and Development , 2017 , 27 ( 3 ): 35 - 38, 43 . (in Chinese)

TAI Y , YANG J , LIU X M . Image super-resolution via deep recursive residual network [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 2790 - 2798 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Honolulu : IEEE , 2017 : 2261 - 2269 .

TONG T , LI G , LIU X J , et al . Image super-resolution using dense skip connections [C]// 2017 IEEE International Conference on Computer Vision(ICCV) . Venice : IEEE , 2017 : 4809 - 4817 .

TAI Y , YANG J , LIU X M , et al . MemNet: A persistent memory network for image restoration [C]// 2017 IEEE International Conference on Computer Vision(ICCV) . Venice : IEEE , 2017 : 4549 - 4557 .

SHAMSOLMOALI P , ZAREAPOOR M , ZHANG J H , et al . Image super resolution by dilated dense progressive network [J]. Image and Vision Computing , 2019 , 88 : 9 - 18 .

PAN Z , LI B , XI T , et al . Real image super resolution via heterogeneous model ensemble using GP-NAS [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Glasgow : Springer , 2020 : 423 - 436 .

JIANG K , WANG Z Y , YI P , et al . Hierarchical dense recursive network for image super-resolution [J]. Pattern Recognition , 2020 , 107 : 107475 .

CHAUDHARI S , MITHAL V , POLATKAN G , et al . An attentive survey of attention models [J]. ACM Transactions on Intelligent Systems and Technology , 2021 , 12 ( 5 ): 53(1 - 32 .

HU J , SHEN L , ALBANIE S , et al . Squeeze-and-excitation networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 8 ): 2011 - 2023 .

ZHANG Y , LI K , LI K , et al . Image super-resolution using very deep residual channel attention networks [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Munich : Springer , 2018 : 294 - 310 .

DAI T , CAI J R , ZHANG Y B , et al . Second-order attention network for single image super-resolution [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 11057 - 11066 .

AHN N , KANG B , Fast SOHN K. , accurate , and lightweight super-resolution with cascading residual network [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Munich : Springer , 2018 : 256 - 272 .

HUI Z , WANG X M , GAO X B . Fast and accurate single image super-resolution via information distillation network [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 723 - 731 .

HUI Z , GAO X , YANG Y , et al . Lightweight image super-resolution with information multi-distillation network [C]// Proceedings of the 27th ACM International Conference on Multimedia . Nice : ACM , 2019 : 2024 - 2032 .

LIU J , TANG J , WU G . Residual feature distillation network for lightweight image super-resolution [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Glasgow : Springer , 2020 : 41 - 55 .

CHU X X , ZHANG B , MA H L , et al . Fast, accurate and lightweight super-resolution with neural architecture search [C]// 2020 25th International Conference on Pattern Recognition(ICPR) . Milan : IEEE , 2021 : 59 - 64 .

LI W B , ZHOU K , QI L , et al . LAPAR: Linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond [C]// Proceedings of the 34th International Conference on Neural Information Processing Systems . Vancouver : ACM , 2020 : 20343 - 20355 .

WANG L G , DONG X Y , WANG Y Q , et al . Exploring sparsity in image super-resolution for efficient inference [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 4915 - 4924 .

RADOSAVOVIC I , KOSARAJU R P , GIRSHICK R , et al . Designing network design spaces [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle : IEEE , 2020 : 10425 - 10433 .

ZHANG K , DANELLJAN M , LI Y , et al . AIM 2020 challenge on efficient super-resolution: methods and results [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Glasgow : Springer , 2020 : 5 - 40 .

GOODFELLOW I J , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems(NIPS 2014) . Montreal : MIT Press , 2014 : 2672 - 2680 .

SAJJADI M S M , SCHÖLKOPF B , HIRSCH M . EnhanceNet: single image super-resolution through automated texture synthesis [C]// 2017 IEEE International Conference on Computer Vision(ICCV) . Venice : IEEE , 2017 : 4501 - 4510 .

WANG X , YU K , WU S , et al . ESRGAN: enhanced super-resolution generative adversarial networks [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Munich : Springer , 2018 : 63 - 79 .

WANG X T , YU K , DONG C , et al . Deep network interpolation for continuous imagery effect transition [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 1692 - 1701 .

SHANG T Z , DAI Q J , ZHU S C , et al . Perceptual extreme super resolution network with receptive field block [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Seattle : IEEE , 2020 : 1778 - 1787 .

ZHANG W L , LIU Y H , DONG C , et al . RankSRGAN: generative adversarial networks with ranker for image super-resolution [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Seoul : IEEE , 2019 : 3096 - 3105 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems . Long Beach : Curran Associates Inc. , 2017 : 6000 - 6010 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16×16 words: Transformers for image recognition at scale [C]// Proceedings of the International Conference on Learning Representations(ICLR 2021) . London : ICLR Press , 2021 : 1 - 22 .

CARION N , MASSA F , SYNNAEVE G , et al . End-to-end object detection with transformers [C]// Proceedings of the European Conference on Computer Vision(ECCV 2020) . Glasgow : Springer , 2020 : 213 - 229 .

ARNAB A , DEHGHANI M , HEIGOLD G , et al . ViViT: A video vision transformer [C]// 2021 IEEE/CVF International Conference on Computer Vision(ICCV) . Montreal : IEEE , 2021 : 6816 - 6826 .

BI J R , ZHU Z L , MENG Q L . Transformer in computer vision [C]// 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology(CEI) . Fuzhou : IEEE , 2021 : 178 - 188 .

LIANG J Y , CAO J Z , SUN G L , et al . SwinIR: image restoration using swin transformer [C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops(ICCVW) . Montreal : IEEE , 2021 : 1833 - 1844 .

LIU Z , LIN Y T , CAO Y , et al . Swin transformer: hierarchical vision transformer using shifted windows [C]// 2021 IEEE/CVF International Conference on Computer Vision(ICCV) . Montreal : IEEE , 2021 : 9992 - 10002 .

LU Z , LIU H , LI J , et al . Efficient transformer for single image super-resolution [EB/OL]. ( 2021-10-28 )[ 2022-01-16 ]. https://arxiv.org/abs/2108.11084 https://arxiv.org/abs/2108.11084 .

SHOCHER A , COHEN N , IRANI M . Zero-shot super-resolution using deep internal learning [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Salt Lake City : IEEE , 2018 : 3118 - 3126 .

BULAT A , YANG J , TZIMIROPOULOS G . To learn image super-resolution, use a GAN to learn how to do image degradation first [C]// Proceedings of the 15th European Conference on Computer Vision(ECCV) . Munich : Springer , 2018 : 187 - 202 .

YUAN Y , LIU S Y , ZHANG J W , et al . Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Salt Lake City : IEEE , 2018 : 814 - 81409 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 2242 - 2251 .

ZHANG Y B , LIU S Y , DONG C , et al . Multiple cycle-in-cycle generative adversarial networks for unsupervised image super-resolution [J]. IEEE Transactions on Image Processing , 2020 , 29 : 1101 - 1112 .

WU S , DONG C , QIAO Y . Blind image restoration based on cycle-consistent network [J/OL]. IEEE Transactions On Multimedia , 2022 . DOI: 10.1109/TMM.2021 http://dx.doi.org/10.1109/TMM.2021 .

3139209 .

TIMOFTE R , ROTHE R , VAN GOOL L . Seven ways to improve example-based single image super resolution [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . Las Vegas : IEEE , 2016 : 1865 - 1873 .

TIMOFTE R , GU S H , WU J Q , et al . NTIRE 2018 challenge on single image super-resolution: Methods and results [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Salt Lake City : IEEE , 2018 : 965 - 96511 .

CAI J R , GU S H , TIMOFTE R , et al . NTIRE 2019 challenge on real image super-resolution: Methods and results [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Long Beach : IEEE , 2019 : 2211 - 2223 .

LUGMAYR A , DANELLJAN M , TIMOFTE R , et al . AIM 2019 challenge on real-world image super-resolution: Methods and results [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW) . Seoul : IEEE , 2019 : 3575 - 3583 .

ZHANG K , GU S H , TIMOFTE R , et al . AIM 2019 challenge on constrained super-resolution: Methods and results [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW) . Seoul : IEEE , 2019 : 3565 - 3574 .

GU S H , DANELLJAN M , TIMOFTE R , et al . AIM 2019 challenge on image extreme super-resolution: Methods and results [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW) . Seoul : IEEE , 2019 : 3556 - 3564 .

ZHANG K , GU S , TIMOFTE R , et al . NTIRE 2020 challenge on perceptual extreme super-resolution: Methods and results [C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Seattle : IEEE , 2020 : 2045 - 2057 .

WEI P , LU H , TIMOFTE R , et al . AIM 2020 challenge on real image super-resolution: methods and results [C]// Proceedings of the European Conference on Computer Vision(ECCV) . Berlin : Springer , 2020 : 392 - 422 .

BHAT G , DANELLJAN M , TIMOFTE R , et al . NTIRE 2021 challenge on burst super-resolution: Methods and results [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Nashville : IEEE , 2021 : 613 - 626 .

IGNATOV A , TIMOFTE R , DENNA M , et al . Real-time quantized image super-resolution on mobile NPUs, mobile AI 2021 challenge: Report [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Nashville : IEEE , 2021 : 2525 - 2534 .

FAN Y C , YU J H , LIU D , et al . Scale-wise convolution for image restoration [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 10770 - 10777 .

YOO J , CHEN Q . SinIR: Efficient general image manipulation with single image reconstruction [C]// Proceedings of the 38th International Conference on Machine Learning(ICML) . Champaign : PMLR , 2021 : 12040 - 12050 .

KONG X T , ZHAO H Y , QIAO Y , et al . ClassSR: A general framework to accelerate super-resolution networks by data characteristic [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Nashville : IEEE , 2021 : 12011 - 12020 .

WANG L G , WANG Y Q , LIN Z P , et al . Learning a single network for scale-arbitrary super-resolution [C]// 2021 IEEE/CVF International Conference on Computer Vision(ICCV) . Montreal : IEEE , 2021 : 4781 - 4790 .

AYAZOGLU M . Extremely lightweight quantization robust real-time single-image super resolution for mobile devices [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW) . Nashville : IEEE , 2021 : 2472 - 2479 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

Cross-CNN: An Animation Cross-Frame Sketch Colorization Algorithm Based on Hybrid Model with CNN and Transformer

No-Reference Screen Content Image Quality Assessment Based on Edge Assistance and Multi-Scale Transformer

MalMKNet: A Multi-Scale Convolutional Neural Network Used for Malware Classification

Related Author

Jing WU

Xiao-jing YE

Feng HUANG

Li-qiong CHEN

Zhi-feng WANG

Wen-xi LIU

WANG Ying

GAO Lan

Related Institution

School of Mechanical Engineering and Automation， Fuzhou University

Advanced Technology Innovation Institute， Fuzhou University

College of Computer and Data Science， Fuzhou University

College of Information Engineering, Capital Normal University

School of Mathematical Science, Capital Normal University

⁰