电子学报 ›› 2022, Vol. 50 ›› Issue (11): 2629-2637.DOI: 10.12263/DZXB.20220155
陈文俊, 杨春玲
收稿日期:
2022-02-07
修回日期:
2022-04-11
出版日期:
2022-11-25
作者简介:
基金资助:
CHEN Wen-jun, YANG Chun-ling
Received:
2022-02-07
Revised:
2022-04-11
Online:
2022-11-25
Published:
2022-11-19
摘要:
现有的图像压缩感知(Image Compressive Sensing, ICS)优化启发网络沿用了传统算法的像素域优化思想,构建了像素域的图像信息流动通道,而没有充分利用卷积神经网络所提取的图像特征中的信息.对此,本文提出了在特征域构建信息流的思想,并设计了一种特征域优化启发ICS网络(Feature-Space Optimization-Inspired Network, FSOINet)以实现该思想.考虑到卷积操作感受野较小,本文通过将自注意力模块引入FSOINet以更高效地利用图像非局部自相似性,进一步提高重构质量,我们将其命名为FSOINet+.此外,本文还提出把迁移学习策略应用于不同采样率图像压缩感知重构网络训练中,提高网络学习效率与重构质量.仿真实验表明,本文所提出的网络在峰值信噪比(Peak Signal to Noise Ratio, PSNR)、结构相似性(Structural Similarity Index Measure, SSIM)与视觉效果上都优于现有的最优ICS重构方法,FSOINet与FSOINet+在Set11数据集上与OPINENet+相比重构图像PSNR分别平均提升了1.04dB和1.27dB.
中图分类号:
陈文俊, 杨春玲. 图像压缩感知的特征域优化及自注意力增强神经网络重构算法[J]. 电子学报, 2022, 50(11): 2629-2637.
CHEN Wen-jun, YANG Chun-ling. Feature-Space Optimization-Inspired and Self-Attention Enhanced Neural Network Reconstruction Algorithm for Image Compressive Sensing[J]. Acta Electronica Sinica, 2022, 50(11): 2629-2637.
方法 | 采样率 | |||||
---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | 平均 | |
GSR | 16.78/0.4520 | 22.79/0.7155 | 26.64/0.8562 | 34.77/0.9466 | 38.76/0.9721 | 27.95/0.7885 |
SCSNet | 21.04/0.5562 | 25.85/0.7839 | 28.52/0.8616 | 34.64/0.9511 | 39.01/0.9769 | 29.81/0.8259 |
CSNet+ | 21.02/0.5566 | 25.86/0.7846 | 28.34/0.8508 | 34.30/0.9490 | 38.52/0.9749 | 29.61/0.8232 |
SPLNet | 21.22/0.5552 | 26.59/0.8177 | 29.49/0.8874 | 35.79/0.9603 | 40.27/0.9815 | 30.67/0.8404 |
OPINENet+ | 20.02/0.5362 | 26.36/0.8186 | 29.81/0.8904 | 36.04/0.9600 | 40.19/0.9800 | 30.48/0.8370 |
BCSNet | 20.81/0.5427 | 26.50/0.7893 | 29.36/0.8650 | 35.40/0.9527 | —— | —— |
AMP-Net | 20.20/0.5425 | 26.17/0.8128 | 29.40/0.8876 | 36.03/0.9623 | 40.34/0.9821 | 30.43/0.8375 |
MADUN | —— | —— | 29.91/0.8986 | 36.94/ | 40.77/0.9832 | —— |
FSOINet* | 21.73/0.5937 | 27.36/0.8415 | 30.44/0.9018 | 37.00/0.9665 | 41.08/0.9832 | 31.52/0.8573 |
FSOINet | ||||||
FSOINet+ | 21.91/0.5984 | 27.47/0.8437 | 30.81/0.9056 | 37.30/0.9678 | 41.29/0.9837 | 31.76/0.8600 |
表1 Set11数据集中各采样率不同算法重构图像PSNR(dB)/SSIM对比
方法 | 采样率 | |||||
---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | 平均 | |
GSR | 16.78/0.4520 | 22.79/0.7155 | 26.64/0.8562 | 34.77/0.9466 | 38.76/0.9721 | 27.95/0.7885 |
SCSNet | 21.04/0.5562 | 25.85/0.7839 | 28.52/0.8616 | 34.64/0.9511 | 39.01/0.9769 | 29.81/0.8259 |
CSNet+ | 21.02/0.5566 | 25.86/0.7846 | 28.34/0.8508 | 34.30/0.9490 | 38.52/0.9749 | 29.61/0.8232 |
SPLNet | 21.22/0.5552 | 26.59/0.8177 | 29.49/0.8874 | 35.79/0.9603 | 40.27/0.9815 | 30.67/0.8404 |
OPINENet+ | 20.02/0.5362 | 26.36/0.8186 | 29.81/0.8904 | 36.04/0.9600 | 40.19/0.9800 | 30.48/0.8370 |
BCSNet | 20.81/0.5427 | 26.50/0.7893 | 29.36/0.8650 | 35.40/0.9527 | —— | —— |
AMP-Net | 20.20/0.5425 | 26.17/0.8128 | 29.40/0.8876 | 36.03/0.9623 | 40.34/0.9821 | 30.43/0.8375 |
MADUN | —— | —— | 29.91/0.8986 | 36.94/ | 40.77/0.9832 | —— |
FSOINet* | 21.73/0.5937 | 27.36/0.8415 | 30.44/0.9018 | 37.00/0.9665 | 41.08/0.9832 | 31.52/0.8573 |
FSOINet | ||||||
FSOINet+ | 21.91/0.5984 | 27.47/0.8437 | 30.81/0.9056 | 37.30/0.9678 | 41.29/0.9837 | 31.76/0.8600 |
数据集 | 方法 | 采样率 | 平均 | ||||
---|---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | |||
BSDS68 | CSNet+ | 21.71/0.5249 | 25.04/0.6845 | 26.89/0.7756 | 31.66/0.9152 | 35.42/0.9614 | 28.14/0.7723 |
SCSNet | 21.88/0.5250 | 24.98/0.6843 | 27.13/0.7785 | 31.76/0.9173 | 35.67/0.9640 | 28.28/0.7738 | |
SPLNet | 22.33/0.5242 | 25.87/0.7198 | 27.85/0.8094 | 32.77/0.9303 | 36.86/0.9708 | 29.13/0.7907 | |
AMP-Net | 22.28/0.5315 | 25.77/0.7204 | 27.85/0.8113 | 32.84/0.9321 | 36.82/0.9715 | 29.11/0.7934 | |
OPINENet+ | 21.88/0.5162 | 25.66/0.7136 | 27.81/0.8040 | 32.50/0.9236 | 36.32/0.9658 | 28.83/0.7846 | |
FSOINet | |||||||
FSOINet+ | 22.83/0.5441 | 26.27/0.7340 | 28.39/0.8210 | 33.37/0.9352 | 37.47/0.9732 | 29.67/0.8015 | |
Urban100 | CSNet+ | 19.27/0.4812 | 22.63/0.6792 | 24.64/0.7741 | 29.90/0.9162 | 33.55/0.9572 | 26.00/0.7616 |
SCSNet | 19.28/0.4798 | 22.63/0.6774 | 24.93/0.7827 | 30.12/0.9193 | 33.92/0.9601 | 26.18/0.7639 | |
SPLNet | 19.55/0.4873 | 23.55/0.7301 | 26.19/0.8290 | 32.11/0.9405 | 36.41/0.9737 | 27.56/0.7921 | |
AMP-Net | 19.62/0.4969 | 23.45/0.7290 | 26.04/0.8283 | 32.19/0.9418 | 36.33/0.9737 | 27.53/0.7939 | |
OPINENet+ | 19.38/0.4872 | 23.70/0.7363 | 26.61/0.8362 | 32.58/0.9414 | 36.62/0.9727 | 27.78/0.7948 | |
FSOINet | |||||||
FSOINet+ | 20.14/0.5331 | 24.80/0.7805 | 28.05/0.8718 | 34.29/0.9569 | 38.31/0.9800 | 29.12/0.8245 |
表2 各采样率不同算法在不同数据集上重构图像PSNR(dB)/SSIM对比
数据集 | 方法 | 采样率 | 平均 | ||||
---|---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | |||
BSDS68 | CSNet+ | 21.71/0.5249 | 25.04/0.6845 | 26.89/0.7756 | 31.66/0.9152 | 35.42/0.9614 | 28.14/0.7723 |
SCSNet | 21.88/0.5250 | 24.98/0.6843 | 27.13/0.7785 | 31.76/0.9173 | 35.67/0.9640 | 28.28/0.7738 | |
SPLNet | 22.33/0.5242 | 25.87/0.7198 | 27.85/0.8094 | 32.77/0.9303 | 36.86/0.9708 | 29.13/0.7907 | |
AMP-Net | 22.28/0.5315 | 25.77/0.7204 | 27.85/0.8113 | 32.84/0.9321 | 36.82/0.9715 | 29.11/0.7934 | |
OPINENet+ | 21.88/0.5162 | 25.66/0.7136 | 27.81/0.8040 | 32.50/0.9236 | 36.32/0.9658 | 28.83/0.7846 | |
FSOINet | |||||||
FSOINet+ | 22.83/0.5441 | 26.27/0.7340 | 28.39/0.8210 | 33.37/0.9352 | 37.47/0.9732 | 29.67/0.8015 | |
Urban100 | CSNet+ | 19.27/0.4812 | 22.63/0.6792 | 24.64/0.7741 | 29.90/0.9162 | 33.55/0.9572 | 26.00/0.7616 |
SCSNet | 19.28/0.4798 | 22.63/0.6774 | 24.93/0.7827 | 30.12/0.9193 | 33.92/0.9601 | 26.18/0.7639 | |
SPLNet | 19.55/0.4873 | 23.55/0.7301 | 26.19/0.8290 | 32.11/0.9405 | 36.41/0.9737 | 27.56/0.7921 | |
AMP-Net | 19.62/0.4969 | 23.45/0.7290 | 26.04/0.8283 | 32.19/0.9418 | 36.33/0.9737 | 27.53/0.7939 | |
OPINENet+ | 19.38/0.4872 | 23.70/0.7363 | 26.61/0.8362 | 32.58/0.9414 | 36.62/0.9727 | 27.78/0.7948 | |
FSOINet | |||||||
FSOINet+ | 20.14/0.5331 | 24.80/0.7805 | 28.05/0.8718 | 34.29/0.9569 | 38.31/0.9800 | 29.12/0.8245 |
方法 | 参数量 | 0.5采样率 | 0.1采样率 | ||
---|---|---|---|---|---|
Set11 | BSDS68 | Set11 | BSDS68 | ||
平均运行时间/s | |||||
SPLNet | 1.388M | 0.0061 | 0.0076 | 0.0090 | 0.0089 |
AMPNet | 1.529M | 0.0562 | 0.0671 | 0.0564 | 0.0649 |
OPINENet+ | 1.095M | 0.0087 | 0.0126 | 0.0134 | 0.0132 |
FSOINet | 1.061M | 0.0198 | 0.0184 | 0.0215 | 0.0190 |
FSOINet+ | 1.086M | 0.0294 | 0.0264 | 0.0283 | 0.0258 |
表3 不同采样率不同算法在Set11与BSDS68上的复杂度对比
方法 | 参数量 | 0.5采样率 | 0.1采样率 | ||
---|---|---|---|---|---|
Set11 | BSDS68 | Set11 | BSDS68 | ||
平均运行时间/s | |||||
SPLNet | 1.388M | 0.0061 | 0.0076 | 0.0090 | 0.0089 |
AMPNet | 1.529M | 0.0562 | 0.0671 | 0.0564 | 0.0649 |
OPINENet+ | 1.095M | 0.0087 | 0.0126 | 0.0134 | 0.0132 |
FSOINet | 1.061M | 0.0198 | 0.0184 | 0.0215 | 0.0190 |
FSOINet+ | 1.086M | 0.0294 | 0.0264 | 0.0283 | 0.0258 |
模型设置 | 测试集PSNR/dB | ||||
---|---|---|---|---|---|
FSIM | DDM | WSEM | Set11 | BSDS68 | Urban100 |
√ | × | × | 40.22 | 36.82 | 36.31 |
× | √ | × | 39.65 | 36.36 | 35.38 |
√ | √ | × | 41.10 | 37.36 | 37.91 |
√ | √ | √ | 41.29 | 37.47 | 38.31 |
表4 0.5采样率下不同模块的消融实验
模型设置 | 测试集PSNR/dB | ||||
---|---|---|---|---|---|
FSIM | DDM | WSEM | Set11 | BSDS68 | Urban100 |
√ | × | × | 40.22 | 36.82 | 36.31 |
× | √ | × | 39.65 | 36.36 | 35.38 |
√ | √ | × | 41.10 | 37.36 | 37.91 |
√ | √ | √ | 41.29 | 37.47 | 38.31 |
训练策略 | 0.01采样率 | 0.1采样率 | 0.3采样率 |
---|---|---|---|
随机初始化训练40 epoch | 21.77/0.5920 | 30.31/0.8995 | 36.74/0.9651 |
随机初始化训练100 epoch | 21.81/0.5940 | 30.51/0.9021 | 36.96/0.9660 |
迁移学习40 epoch | 21.88/0.5958 | 30.57/0.9020 | 37.00/0.9664 |
表5 不同训练策略的FSOINet在Set11上的重构图像PSNR(dB)/SSIM
训练策略 | 0.01采样率 | 0.1采样率 | 0.3采样率 |
---|---|---|---|
随机初始化训练40 epoch | 21.77/0.5920 | 30.31/0.8995 | 36.74/0.9651 |
随机初始化训练100 epoch | 21.81/0.5940 | 30.51/0.9021 | 36.96/0.9660 |
迁移学习40 epoch | 21.88/0.5958 | 30.57/0.9020 | 37.00/0.9664 |
1 | DONOHO D L. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306. |
2 | DUARTE M F, DAVENPORT M A, TAKHAR D, et al. Single-pixel imaging via compressive sampling[J]. IEEE Signal Processing Magazine, 2008, 25(2): 83-91. |
3 | ZHU L, WU X, SUN Z, et al. Compressed-sensing accelerated 3-dimensional magnetic resonance cholangiopancreatography: application in suspected pancreatic diseases[J]. Investigative Radiology, 2018, 53(3): 150-157. |
4 | YUAN X, BRADY D J, KATSAGGELOS A K. Snapshot compressive imaging: theory, algorithms, and applications[J]. IEEE Signal Processing Magazine, 2021, 38(2): 65-88. |
5 | 禤韵怡, 杨春玲. 基于帧间组稀疏的两阶段递归增强视频压缩感知重构网络[J]. 电子学报, 2021, 49(3): 435-442. |
XUAN Yun-yi, YANG Chun-ling. Two-stage recursive enhancement reconstruction based on video inter-frame group sparse representation in compressed video sensing[J]. Acta Electronica Sinica, 2021, 49(3): 435-442. (in Chinese) | |
6 | GAN L. Block compressed sensing of natural images[C]//15th International Conference on Digital Signal Processing. Cardiff: IEEE, 2007: 403-406. |
7 | CHEN C, TRAMEL E W, FOWLER J E. Compressed-sensing recovery of images and video using multihypothesis predictions[C]//2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers. Pacific Grove: IEEE, 2011: 1193-1198. |
8 | ZHANG J, ZHAO D, GAO W. Group-based sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2014, 23(8): 3336-3351. |
9 | KULKARNI K, LOHIT S, TURAGA P, et al. Reconnet: non-iterative reconstruction of images from compressively sensed measurements[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 449-458. |
10 | SHI W, JIANG F, ZHANG S, et al. Deep networks for compressed image sensing[C]//2017 IEEE International Conference on Multimedia and Expo. Hong Kong: IEEE, 2017: 877-882. |
11 | SHI W, JIANG F, LIU S, et al. Scalable convolutional neural network for image compressed sensing[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 12290-12299. |
12 | SHI W, JIANG F, LIU S, et al. Image compressed sensing using convolutional neural network[J]. IEEE Transactions on Image Processing, 2020, 29: 375-388. |
13 | CHEN J, SUN Y, LIU Q, et al. Learning memory augmented cascading network for compressed sensing of images[C]//2020 European Conference on Computer Vision. Glasgow: Springer, 2020: 513-529. |
14 | ZHOU S, HE Y, LIU Y, et al. Multi-channel deep networks for block-based image compressive sensing[J]. IEEE Transactions on Multimedia, 2021, 23: 2627-2640. |
15 | ZHANG J, GHANEM B. ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1828-1837. |
16 | 裴翰奇, 杨春玲, 魏志超, 曹燕. 基于SPL迭代思想的图像压缩感知重构神经网络[J]. 电子学报, 2021, 49(6): 1195-1203. |
PEI Han-qi, YANG Chun-ling, WEI Zhi-chao, CAO Yan. Image compressive sensing reconstruction network based on iterative SPL theory[J]. Acta Electronica Sinica, 2021, 49(6): 1195-1203. (in Chinese) | |
17 | ZHANG Z, LIU Y, LIU J, et al. AMP-Net: denoising-based deep unfolding for compressive image sensing[J]. IEEE Transactions on Image Processing, 2021, 30: 1487-1500. |
18 | ZHANG J, ZHAO C, GAO W. Optimization-inspired compact deep compressive sensing[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(4): 765-774. |
19 | SONG J, CHEN B, ZHANG J. Memory-augmented deep unfolding network for compressive sensing[C]//2021 ACM International Conference on Multimedia. Chengdu: ACM, 2021: 4249-4258. |
20 | D'ASCOLI S, Touvron H, Leavitt M L, et al. Convit: improving vision transformers with soft convolutional inductive biases[C]//2021 International Conference on Machine Learning. Virtual Only: ACM, 2021: 2286-2296. |
21 | ARBEL'AEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 898-916. |
22 | KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. [2022-04-11]. . |
23 | LOSHCHILOV I, HUTTER F. Sgdr: Stochastic gradient descent with warm restarts[EB/OL]. [2022-04-11]. . |
24 | MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//2001 IEEE International Conference on Computer Vision. Vancouver: IEEE, 2001: 416-423. |
25 | HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 5197-5206. |
26 | CHEN W, YANG C, YANG X. FSOINET: feature-space optimization-inspired network for image compressive sensing[EB/OL]. [2022-04-11]. . |
[1] | 吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294. |
[2] | 毛国君, 王者浩, 黄山, 王翔. 基于剪边策略的图残差卷积深层网络模型[J]. 电子学报, 2022, 50(9): 2205-2214. |
[3] | 李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089. |
[4] | 琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126. |
[5] | 袁海英, 曾智勇, 成君鹏. 面向灵活并行度的稀疏卷积神经网络加速器[J]. 电子学报, 2022, 50(8): 1811-1818. |
[6] | 张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN:用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858. |
[7] | 张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893. |
[8] | 王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936. |
[9] | 徐兴荣, 刘聪, 李婷, 郭娜, 任崇广, 曾庆田. 基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法[J]. 电子学报, 2022, 50(8): 1975-1984. |
[10] | 邵志文, 周勇, 谭鑫, 马利庄, 刘兵, 姚睿. 基于深度学习的表情动作单元识别综述[J]. 电子学报, 2022, 50(8): 2003-2017. |
[11] | 裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547. |
[12] | 张志文, 刘天歌, 聂鹏举. 基于实景数据增强和双路径融合网络的实时街景语义分割算法[J]. 电子学报, 2022, 50(7): 1609-1620. |
[13] | 欧阳与点, 谢鲲, 谢高岗, 文吉刚. 面向大规模网络测量的数据恢复算法:基于关联学习的张量填充[J]. 电子学报, 2022, 50(7): 1653-1663. |
[14] | 王相海, 赵晓阳, 王鑫莹, 赵克云, 宋传鸣. 非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J]. 电子学报, 2022, 50(7): 1753-1765. |
[15] | 彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||