非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建
Single Image Super-Resolution Reconstruction Using Deep Residual Networks with Non-decimated Wavelet Edge Learning
- 电子学报 2022年50卷第7期 页码：1753-1765
- 作者机构：
1.辽宁师范大学地理科学学院，辽宁大连 116029
2.辽宁师范大学计算机科学与信息技术学院，辽宁大连 116081
- 作者简介：
[ "王相海 男，1965年11月出生于吉林省汪清县.现为辽宁师范大学地理科学学院和计算机与信息技术学院教授、博士生导师.主要研究方向为多媒体信息处理、遥感影像信息处理.E-mail： xhwang@lnnu.edu.cn" ]
[ "赵晓阳 女，1994年4月出生于辽宁省沈阳市. 现为辽宁师范大学地理科学学院博士研究生. 主要研究方向为深度学习与遥感影像智能信息处理.E-mail: zxy_lnnu@163.com" ]
[ "王鑫莹 女，1995年10月出生于黑龙江省齐齐哈尔市. 现为辽宁师范大学计算机与信息技术学院硕士研究生. 主要研究方向为遥感图像处理.E-mail: 1603331385@qq.com" ]
[ "赵克云 男，1997年11月出生于辽宁省沈阳市. 现为辽宁师范大学计算机与信息技术学院硕士研究生. 主要研究方向为深度学习与遥感图像处理.E-mail: 1184234775@qq.com" ]
[ "宋传鸣 男，1980年10月出生于辽宁省沈阳市. 现为辽宁师范大学计算机与信息技术学院教授. 主要研究方向为多媒体信息处理和智能图像处理.E-mail: chmsong@163.com" ]
- 基金信息：国家自然科学基金(41971388);辽宁省高等学校创新团队支持计划(LT2017013)
DOI：10.12263/DZXB.20210854
中图分类号： TP391收稿：2021-07-06，
修回：2022-03-29，
纸质出版：2022-07-25
- 稿件说明：
移动端阅览
王相海,赵晓阳,王鑫莹等.非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J].电子学报,2022,50(07):1753-1765.
WANG Xiang-hai,ZHAO Xiao-yang,WANG Xin-ying,et al.Single Image Super-Resolution Reconstruction Using Deep Residual Networks with Non-decimated Wavelet Edge Learning[J].ACTA ELECTRONICA SINICA,2022,50(07):1753-1765.
王相海,赵晓阳,王鑫莹等.非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J].电子学报,2022,50(07):1753-1765. DOI： 10.12263/DZXB.20210854.
WANG Xiang-hai,ZHAO Xiao-yang,WANG Xin-ying,et al.Single Image Super-Resolution Reconstruction Using Deep Residual Networks with Non-decimated Wavelet Edge Learning[J].ACTA ELECTRONICA SINICA,2022,50(07):1753-1765. DOI： 10.12263/DZXB.20210854.
摘要
图像超分辨率重建作为一个典型的非适定问题一直受到重视，尽管近年来出现了许多行之有效的卷积神经网络超分辨率重建模型，但如何全面挖掘图像先验信息，用以提高重建图像的细节清晰度仍有待深入研究.本文提出一种基于非抽取Wavelet变换的边缘学习深度残差网络单幅图像超分辨重建模型NDW-EDRN（Non-Decimated Wavelet Edge learning using Deep Residual Networks），在图像经非抽取Wavelet变换后获得多冗余信息、平滑及梯度值较小的低频区域和边缘及梯度值较大的高频区域的基础上，将整体网络框架设计为采用不同结构的CNN（Convolutional Neural Networks）模型来对低频子带与高频子带分别进行学习的策略：对低频子带采用稠密跳跃连接的方式整体性学习低频子带间的映射关系；对高频子带采用一种新型的U-net模型，将图像退化过程中所丢失的边缘作为网络的期望输出，通过基于块的跳跃连接来使网络更精细地学习缺失性边缘，从而更加充分、有效地获取图像在退化过程中所丢失的边缘细节信息.大量实验结果表明，该网络模型能够有效提高重建图像的质量，特别在恢复低分辨率图像的边缘信息方面具有一定的优势，在一定程度上弥补了传统CNN网络模型捕捉图像细节信息的不足.
Abstract
As a typical ill-posed problem
image super-resolution reconstruction has always been paid attention to. Although many effective super-resolution reconstruction models using convolutional neural networks have been developed in recent years
how to mine the prior information in the image entirely to improve the details in reconstructed image needs to be further studied. In this paper
a single image super-resolution reconstruction model using deep residual networks with non-decimated wavelet transform edge learning NDW-EDRN(Non-Decimated Wavelet Edge learning using Deep Residual Networks)is proposed. On the basis of low-frequency regions with multiple redundant information
smoothness and low gradient values
as well as high-frequency regions with edges and high gradient values that are obtained from an image after a non-decimated wavelet transform
CNN(Convolutional Neural Networks) model with different structures are designed to learn the low and high-frequency subbands separately in the overall network framework: dense skip connection is introduced for integrating learning the mapping between low-frequency subbands; a novel U-net model is designed
which makes the edges that lost in the process of image degradation as the expected output of the network
and block-based skip connection is designed at the same time for making the network learn the lost edges more robust
to obtain the lost edge details during image degradation more sufficiently and comprehensively. A large number of experimental results show that the network can improve the quality of the reconstructed image effectively and especially has a certain advantage in recovering low-resolution images lost
and makes up for the deficiencies of traditional CNN models in image learning detail information to some extent.
关键词
Keywords
references
LECUN Y , BOSER B E , DENKER J S , et al . Backpropagation applied to handwritten zip code recognition [J]. Neural Computation , 1989 , 1 ( 4 ): 541 - 551 .
LECUN Y , BOTTOU L . Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .
周飞燕 , 金林鹏 , 董军 . 卷积神经网络研究综述 [J]. 计算机学报 , 2017 , 40 ( 6 ): 1229 - 1251 .
Zhou Fei-yan , Jin Lin-peng , Dong Jun . Review of convolutional neural network [J]. Chinese Journal of Computers , 2017 , 40 ( 6 ): 1229 - 1251 . (in Chinese)
KHAN S , RAHMANI H , SHAH S A A , et al . A guide to convolutional neural networks for computer vision [J]. Synthesis Lectures on Computer Vision , 2018 , 8 ( 1 ): 1 - 207 .
DONG C , LOY C C , HE K , TANG X . Learning a deep convolutional network for image super-resolution [C]// Proceedings of the 13th European Conference on Computer Vision . Berlin : Springer , 2014 : 184 - 199 .
DONG C , LOY C C , TANG X . Accelerating the super-resolution convolutional neural network [C]// Proceedings of the European Conference on Computer Vision . Berlin : Springer , 2016 : 391 - 407 .
KIM J , LEE J K , LEE K M . Accurate image super-resolution using very deep convolutional networks [C]// Proceedings of the Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 1646 - 1654 .
KIM J , LEE J K , LEE K M . Deeply-recursive convolutional network for image super-resolution [C]// Proceedings of the Computer Vision and Pattern Recognition . Piscataway, NJ : IEEE , 2016 : 1637 - 1645 .
SHI W Z , CABALLERO J , HUSZAR F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]// Proceedings of the Computer Vision and Pattern Recognition . Piscataway, NJ : IEEE , 2016 : 1874 - 1883 .
LAI W S , HUANG J B , AHUJA N , et al . Fast and accurate image super-resolution with deep laplacian pyramid networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 41 ( 11 ): 2599 - 2613 .
CHRISTIAN L , LUCAS T , FERENC H , et al . Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 105 - 114 .
WU B , DUAN H , LIU Z , et al . SRPGAN: Perceptual generative adversarial network for single image super resolution [J/OL]. ( 2017-12-20 ) [ 2021-06-20 ]. https:// arxiv.org /abs/1712.05927 https://arxiv.org/abs/1712.05927 .
WANG X , YU K , WU S , et al . Esrgan: Enhanced super-resolution generative adversarial networks [C]// Proceedings of the European Conference on Computer Vision . Munich : Springer , 2018 : 63 - 79 .
JOLICOEUR-MARTINEAU A . The relativistic discriminator: A key element missing from standard GAN [J/OL]. ( 2018-09-10 )[ 2021-06-25 ]. https://arxiv.org/abs/1807. 00734 https://arxiv.org/abs/1807.00734 .
宋传鸣 , 赵长伟 , 刘丹 , 王相海 . 3D多尺度几何分析研究进展 [J]. 软件学报 , 2015 , 26 ( 5 ): 1213 - 1236 .
Song Chuan-ming , Zhao Chang-wei , Liu Dan , Wang Xiang-hai . Advances in three-dimensional multiscale geometrical analysis [J]. Journal of Software , 2015 , 26 ( 5 ): 1213 - 1236 . (in Chinese)
VYAS A , YU S , PAIK J . Multiscale Transforms with Application to Image Processing: Wavelets and Wavelet transform [M]. Singapore : Springer , 2018 .
BAE W , YOO J , YE J C . Beyond deep residual learning for image restoration: Persistent homology-guided manifold simplification [C]// Proceedings of the Computer Vision and Pattern Recognition Workshops . Honolulu : IEEE , 2017 : 1141 - 1149 .
GUO T , MOUSAVI H S , VU T H , et al . Deep wavelet prediction for image super-resolution [C]// Proceedings of the Computer Vision and Pattern Recognition Workshops . Honolulu : IEEE , 2017 : 1100 - 1109 .
YUE L , SHEN H , LI J , et al . Image super-resolution: The techniques, applications, and future [J]. Signal processing , 2016 , 128 ( 11 ): 389 - 408 .
SHENSA M J . The discrete wavelet transform: Wedding the atrous and mallat algorithms [J]. IEEE Transactions on Signal Processing , 1992 , 40 ( 10 ): 2464 - 2482 .
NASON G P , SILVERMAN B W . The stationary wavelet transform and some statistical applications [C]// Wavelets and Statistics Lecture Notes in Statistics . Berlin : Springer , 1995 : 281 - 300 .
NASON G P . Wavelet methods in statistics with R [J]. Journal of the Royal Statistical Society , 2010 , 173 ( 1 ): 273 - 273 .
DAUBECHIES I . Orthonormal bases of compactly supported wavelets [J]. Comms Pure Appl Math , 1988 , 41 ( 7 ): 909 - 996 .
BEVILACQUA M , ROUMY A , GUILLEMOT C , et al . Low-complexity single image super-resolution based on nonnegative neighbor embedding [C]// Proceedings of the British Machine Vision Conference . Guildford, UK : BMVA Press , 2012 : 135 - 144 .
XIN D , REN Y , MAI X , et al . Wavelet domain style transfer for an effective perception-distortion tradeoff in single image super-resolution [C]// Proceedings of the IEEE International Conference on Computer Vision . Seoul : IEEE , 2019 : 3076 - 3085 .
TONG T , LI G , LIU X , et al . Image super-resolution using dense skip connections [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 4809 - 4817 .
TONG T , LI G , LIU X , et al . Image super-resolution using dense skip connections [C]// Proceedings of the IEEE International Conference on Computer Vision . Piscataway, NJ : IEEE , 2017 : 4809 - 4817 .
RONNEBERGER O , FISCHER P , BROX T . U-Net: Convolutional networks for biomedical image segmentation [C]// Proceedings of the International Conference on Medical Image Computing & Computer-assisted Intervention . Munich : Springer , 2015 : 234 - 241 .
RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C]// Proceedings of the Medical Image Computing and Computer-Assisted Intervention . Berlin : Springer , 2015 : 234 - 241 .
HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C]// Proceedings of the Computer Vision and Pattern Recognition . Las Vegas : IEEE , 2016 : 770 - 778 .
ARBELAEZ P , MAIRE M , FOWLKES C , et al . Contour detection and hierarchical image segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 5 ): 898 - 916 .
YAGN J , WEIGHT J , HUANG T S , MA Y . Image super-resolution via sparse representation [J]. IEEE Transactions Image Processing , 2010 , 19 ( 11 ): 2861 - 2873 .
ZEYDE R , ELAD M , PROTTER M . On single image scale-up using sparse-representations [C]// Proceedings of the 7th International Conference Curves Surfaces . Berlin : Springer , 2012 : 11 - 730 .
HUANG J B , SINGH A , AHUJA N . Single image super-resolution from transformed self-exemplars [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston : IEEE , 2015 : 5197 - 5206 .
KINGMA D , BA J . Adam: A method for stochastic optimization [J/OL]. ( 2017-01-17 )[ 2021-05-20 ]. https://arxiv.org /abs/1412.6980 https://arxiv.org/abs/1412.6980 .
ZHANG K , ZUO W , ZHANG L . Learning a single convolutional super-resolution network for multiple degradations [C]// Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition . Salt Lake City : IEEE , 2018 : 3262 - 3271 .
HAN W , CHANG S , LIU D , et al . Image super-resolution via dual-state recurrent networks [C]// Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition . Salt Lake City : IEEE , 2018 : 1654 - 1663 .
TAI Y , YANG J , LIU X . Image super-resolution via deep recursive residual network [C]// Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition . Honolulu : IEEE , 2017 : 2790 - 2798 .
TAI Y , YANG J , LIU X , XU C . Memnet: A persistent memory network for image restoration [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 4549 - 4557 .
0
浏览量
8
下载量
2
CSCD
- 网络PDF下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies