电子学报 ›› 2022, Vol. 50 ›› Issue (11): 2629-2637.DOI: 10.12263/DZXB.20220155

• 学术论文 • 上一篇    下一篇

图像压缩感知的特征域优化及自注意力增强神经网络重构算法

陈文俊, 杨春玲   

  1. 华南理工大学电子与信息学院,广东 广州 510640
  • 收稿日期:2022-02-07 修回日期:2022-04-11 出版日期:2022-11-25
    • 作者简介:
    • 陈文俊 男,1997年出生,福建福州人.现为华南理工大学电子与信息学院硕士研究生.主要研究方向为图像/视频压缩感知.E‑mail: eecwjjun@mail.scut.edu.cn
      杨春玲(通讯作者) 女,1970年出生,河南新乡人.现为华南理工大学电子与信息学院博士生导师.主要研究方向为图像/视频压缩编码、图像质量评价.
    • 基金资助:
    • 广东省自然科学基金 (2019A1515011949)

Feature-Space Optimization-Inspired and Self-Attention Enhanced Neural Network Reconstruction Algorithm for Image Compressive Sensing

CHEN Wen-jun, YANG Chun-ling   

  1. School of Electronic and Information Engineering,South China University of Technology,Guangzhou,Guangdong 510640,China
  • Received:2022-02-07 Revised:2022-04-11 Online:2022-11-25 Published:2022-11-19

摘要:

现有的图像压缩感知(Image Compressive Sensing, ICS)优化启发网络沿用了传统算法的像素域优化思想,构建了像素域的图像信息流动通道,而没有充分利用卷积神经网络所提取的图像特征中的信息.对此,本文提出了在特征域构建信息流的思想,并设计了一种特征域优化启发ICS网络(Feature-Space Optimization-Inspired Network, FSOINet)以实现该思想.考虑到卷积操作感受野较小,本文通过将自注意力模块引入FSOINet以更高效地利用图像非局部自相似性,进一步提高重构质量,我们将其命名为FSOINet+.此外,本文还提出把迁移学习策略应用于不同采样率图像压缩感知重构网络训练中,提高网络学习效率与重构质量.仿真实验表明,本文所提出的网络在峰值信噪比(Peak Signal to Noise Ratio, PSNR)、结构相似性(Structural Similarity Index Measure, SSIM)与视觉效果上都优于现有的最优ICS重构方法,FSOINet与FSOINet+在Set11数据集上与OPINENet+相比重构图像PSNR分别平均提升了1.04dB和1.27dB.

关键词: 图像压缩感知, 深度学习, 卷积神经网络, 自注意力, 图像重构, 迁移学习

Abstract:

The existing optimization-inspired networks for image compressive sensing(ICS) implement information optimization and flow in the pixel domain following the traditional algorithms, which does not make full use of the information in the image feature maps extracted by the convolutional neural network. This paper proposes the idea of constructing information flow in the feature domain. A feature-space optimization-inspired network(FSOINet) is designed to implement this idea. Considering the small receptive field of the convolution operation, this paper introduces the self-attention module into FSOINet to efficiently utilize the non-local self-similarity of images to further improve the reconstruction quality, which is named FSOINet+. In addition, this paper proposes a training strategy that applies transfer learning to the ICS reconstruction network training for different sampling rates to improve the network learning efficiency and reconstruction quality. Experimental results show that the proposed method is superior to the existing state-of-the-art ICS methods in peak signal to noise ratio(PSNR), structural similarity index measure(SSIM) and the visual effect. Compared with OPINENet+ on the Set11 dataset, FSOINet and FSOINet+ have an average PSNR improvement of 1.04dB/1.27dB respectively.

Key words: image compressive sensing, deep learning, convolutional neural networks, self-attention, image reconstruction, transfer learning

中图分类号: