电子学报 ›› 2022, Vol. 50 ›› Issue (7): 1753-1765.DOI: 10.12263/DZXB.20210854

• 学术论文 • 上一篇    

非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建

王相海1,2, 赵晓阳1, 王鑫莹2, 赵克云2, 宋传鸣2   

  1. 1.辽宁师范大学地理科学学院,辽宁 大连 116029
    2.辽宁师范大学计算机科学与信息技术学院,辽宁 大连 116081
  • 收稿日期:2021-07-06 修回日期:2022-03-29 出版日期:2022-07-25 发布日期:2022-07-30
  • 作者简介:王相海 男,1965年11月出生于吉林省汪清县.现为辽宁师范大学地理科学学院和计算机与信息技术学院教授、博士生导师.主要研究方向为多媒体信息处理、遥感影像信息处理.E-mail: xhwang@lnnu.edu.cn
    赵晓阳 女,1994年4月出生于辽宁省沈阳市. 现为辽宁师范大学地理科学学院博士研究生. 主要研究方向为深度学习与遥感影像智能信息处理.E-mail: zxy_lnnu@163.com
    王鑫莹 女,1995年10月出生于黑龙江省齐齐哈尔市. 现为辽宁师范大学计算机与信息技术学院硕士研究生. 主要研究方向为遥感图像处理.E-mail: 1603331385@qq.com
    赵克云 男,1997年11月出生于辽宁省沈阳市. 现为辽宁师范大学计算机与信息技术学院硕士研究生. 主要研究方向为深度学习与遥感图像处理.E-mail: 1184234775@qq.com
    宋传鸣 男,1980年10月出生于辽宁省沈阳市. 现为辽宁师范大学计算机与信息技术学院教授. 主要研究方向为多媒体信息处理和智能图像处理.E-mail: chmsong@163.com
  • 基金资助:
    国家自然科学基金(41971388);辽宁省高等学校创新团队支持计划(LT2017013)

Single Image Super-Resolution Reconstruction Using Deep Residual Networks with Non-decimated Wavelet Edge Learning

WANG Xiang-hai1,2, ZHAO Xiao-yang1, WANG Xin-ying2, ZHAO Ke-yun2, SONG Chuan-ming2   

  1. 1.School of Geographical Science,Liaoning Normal University,Dalian,Liaoning 116029,China
    2.School of Computer and Information Technology,Liaoning Normal University,Dalian,Liaoning 116081,China
  • Received:2021-07-06 Revised:2022-03-29 Online:2022-07-25 Published:2022-07-30

摘要:

图像超分辨率重建作为一个典型的非适定问题一直受到重视,尽管近年来出现了许多行之有效的卷积神经网络超分辨率重建模型,但如何全面挖掘图像先验信息,用以提高重建图像的细节清晰度仍有待深入研究.本文提出一种基于非抽取Wavelet变换的边缘学习深度残差网络单幅图像超分辨重建模型NDW-EDRN(Non-Decimated Wavelet Edge learning using Deep Residual Networks),在图像经非抽取Wavelet变换后获得多冗余信息、平滑及梯度值较小的低频区域和边缘及梯度值较大的高频区域的基础上,将整体网络框架设计为采用不同结构的CNN(Convolutional Neural Networks)模型来对低频子带与高频子带分别进行学习的策略:对低频子带采用稠密跳跃连接的方式整体性学习低频子带间的映射关系;对高频子带采用一种新型的U-net模型,将图像退化过程中所丢失的边缘作为网络的期望输出,通过基于块的跳跃连接来使网络更精细地学习缺失性边缘,从而更加充分、有效地获取图像在退化过程中所丢失的边缘细节信息.大量实验结果表明,该网络模型能够有效提高重建图像的质量,特别在恢复低分辨率图像的边缘信息方面具有一定的优势,在一定程度上弥补了传统CNN网络模型捕捉图像细节信息的不足.

关键词: 卷积神经网络, 残差学习, 非抽取小波变换, 图像超分辨率重建, 纹理边缘信息

Abstract:

As a typical ill-posed problem, image super-resolution reconstruction has always been paid attention to. Although many effective super-resolution reconstruction models using convolutional neural networks have been developed in recent years, how to mine the prior information in the image entirely to improve the details in reconstructed image needs to be further studied. In this paper, a single image super-resolution reconstruction model using deep residual networks with non-decimated wavelet transform edge learning NDW-EDRN(Non-Decimated Wavelet Edge learning using Deep Residual Networks)is proposed. On the basis of low-frequency regions with multiple redundant information, smoothness and low gradient values, as well as high-frequency regions with edges and high gradient values that are obtained from an image after a non-decimated wavelet transform, CNN(Convolutional Neural Networks) model with different structures are designed to learn the low and high-frequency subbands separately in the overall network framework: dense skip connection is introduced for integrating learning the mapping between low-frequency subbands; a novel U-net model is designed, which makes the edges that lost in the process of image degradation as the expected output of the network, and block-based skip connection is designed at the same time for making the network learn the lost edges more robust, to obtain the lost edge details during image degradation more sufficiently and comprehensively. A large number of experimental results show that the network can improve the quality of the reconstructed image effectively and especially has a certain advantage in recovering low-resolution images lost, and makes up for the deficiencies of traditional CNN models in image learning detail information to some extent.

Key words: convolutional neural networks, residual learning, non-decimated wavelet transform, image super-resolution reconstruction, texture edge information

中图分类号: