• • 上一篇
陈文俊, 杨春玲
收稿日期:
2022-02-07
修回日期:
2022-04-11
作者简介:
基金资助:
CHEN Wen-jun, YANG Chun-ling
Received:
2022-02-07
Revised:
2022-04-11
摘要:
现有的图像压缩感知(Image Compressive Sensing, ICS)优化启发网络沿用了传统算法的像素域优化思想,构建了像素域的图像信息流动通道,而没有充分利用卷积神经网络所提取的图像特征中的信息.对此,本文提出了在特征域构建信息流的思想,并设计了一种特征域优化启发ICS网络(Feature-Space Optimization-Inspired Network, FSOINet)以实现该思想.考虑到卷积操作感受野较小,本文通过将自注意力模块引入FSOINet以更高效地利用图像非局部自相似性,进一步提高重构质量,我们将其命名为FSOINet+.此外,本文还提出把迁移学习策略应用于不同采样率图像压缩感知重构网络训练中,提高网络学习效率与重构质量.仿真实验表明,本文所提出的网络在峰值信噪比(Peak Signal to Noise Ratio, PSNR)、结构相似性(Structural Similarity Index Measure, SSIM)与视觉效果上都优于现有的最优ICS重构方法,FSOINet与FSOINet+在Set11数据集上与OPINENet+相比重构图像PSNR分别平均提升了1.04dB和1.27dB.
中图分类号:
陈文俊, 杨春玲. 图像压缩感知的特征域优化及自注意力增强神经网络重构算法[J]. 电子学报, DOI: 10.12263/DZXB.20220155.
CHEN Wen-jun, YANG Chun-ling. Feature-Space Optimization-Inspired and Self-Attention Enhanced Neural Network Reconstruction Algorithm for Image Compressive Sensing[J]. Acta Electronica Sinica, DOI: 10.12263/DZXB.20220155.
采样率方法 | 0.01 | 0.05 | 0.1 | 0.3 | 0.5 | 平均 |
---|---|---|---|---|---|---|
GSR | 16.78/0.4520 | 22.79/0.7155 | 26.64/0.8562 | 34.77/0.9466 | 38.76/0.9721 | 27.95/0.7885 |
SCSNet | 21.04/0.5562 | 25.85/0.7839 | 28.52/0.8616 | 34.64/0.9511 | 39.01/0.9769 | 29.81/0.8259 |
CSNet+ | 21.02/0.5566 | 25.86/0.7846 | 28.34/0.8508 | 34.30/0.9490 | 38.52/0.9749 | 29.61/0.8232 |
SPLNet | 21.22/0.5552 | 26.59/0.8177 | 29.49/0.8874 | 35.79/0.9603 | 40.27/0.9815 | 30.67/0.8404 |
OPINENet+ | 20.02/0.5362 | 26.36/0.8186 | 29.81/0.8904 | 36.04/0.9600 | 40.19/0.9800 | 30.48/0.8370 |
BCSNet | 20.81/0.5427 | 26.50/0.7893 | 29.36/0.8650 | 35.40/0.9527 | —— | —— |
AMP-Net | 20.20/0.5425 | 26.17/0.8128 | 29.40/0.8876 | 36.03/0.9623 | 40.34/0.9821 | 30.43/0.8375 |
MADUN | —— | —— | 29.91/0.8986 | 36.94/ | 40.77/0.9832 | —— |
FSOINet* | 21.73/0.5937 | 27.36/0.8415 | 30.44/0.9018 | 37.00/0.9665 | 41.08/0.9832 | 31.52/0.8573 |
FSOINet | ||||||
FSOINet+ | 21.91/0.5984 | 27.47/0.8437 | 30.81/0.9056 | 37.30/0.9678 | 41.29/0.9837 | 31.76/0.8600 |
表1 Set11数据集中各采样率不同算法重构图像PSNR(dB)/SSIM对比
采样率方法 | 0.01 | 0.05 | 0.1 | 0.3 | 0.5 | 平均 |
---|---|---|---|---|---|---|
GSR | 16.78/0.4520 | 22.79/0.7155 | 26.64/0.8562 | 34.77/0.9466 | 38.76/0.9721 | 27.95/0.7885 |
SCSNet | 21.04/0.5562 | 25.85/0.7839 | 28.52/0.8616 | 34.64/0.9511 | 39.01/0.9769 | 29.81/0.8259 |
CSNet+ | 21.02/0.5566 | 25.86/0.7846 | 28.34/0.8508 | 34.30/0.9490 | 38.52/0.9749 | 29.61/0.8232 |
SPLNet | 21.22/0.5552 | 26.59/0.8177 | 29.49/0.8874 | 35.79/0.9603 | 40.27/0.9815 | 30.67/0.8404 |
OPINENet+ | 20.02/0.5362 | 26.36/0.8186 | 29.81/0.8904 | 36.04/0.9600 | 40.19/0.9800 | 30.48/0.8370 |
BCSNet | 20.81/0.5427 | 26.50/0.7893 | 29.36/0.8650 | 35.40/0.9527 | —— | —— |
AMP-Net | 20.20/0.5425 | 26.17/0.8128 | 29.40/0.8876 | 36.03/0.9623 | 40.34/0.9821 | 30.43/0.8375 |
MADUN | —— | —— | 29.91/0.8986 | 36.94/ | 40.77/0.9832 | —— |
FSOINet* | 21.73/0.5937 | 27.36/0.8415 | 30.44/0.9018 | 37.00/0.9665 | 41.08/0.9832 | 31.52/0.8573 |
FSOINet | ||||||
FSOINet+ | 21.91/0.5984 | 27.47/0.8437 | 30.81/0.9056 | 37.30/0.9678 | 41.29/0.9837 | 31.76/0.8600 |
数据集 | 方法 | 采样率 | 平均 | ||||
---|---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | |||
BSDS68 | CSNet+ | 21.71/0.5249 | 25.04/0.6845 | 26.89/0.7756 | 31.66/0.9152 | 35.42/0.9614 | 28.14/0.7723 |
SCSNet | 21.88/0.5250 | 24.98/0.6843 | 27.13/0.7785 | 31.76/0.9173 | 35.67/0.9640 | 28.28/0.7738 | |
SPLNet | 22.33/0.5242 | 25.87/0.7198 | 27.85/0.8094 | 32.77/0.9303 | 36.86/0.9708 | 29.13/0.7907 | |
AMP-Net | 22.28/0.5315 | 25.77/0.7204 | 27.85/0.8113 | 32.84/0.9321 | 36.82/0.9715 | 29.11/0.7934 | |
OPINENet+ | 21.88/0.5162 | 25.66/0.7136 | 27.81/0.8040 | 32.50/0.9236 | 36.32/0.9658 | 28.83/0.7846 | |
FSOINet | |||||||
FSOINet+ | 22.83/0.5441 | 26.27/0.7340 | 28.39/0.8210 | 33.37/0.9352 | 37.47/0.9732 | 29.67/0.8015 | |
Urban100 | CSNet+ | 19.27/0.4812 | 22.63/0.6792 | 24.64/0.7741 | 29.90/0.9162 | 33.55/0.9572 | 26.00/0.7616 |
SCSNet | 19.28/0.4798 | 22.63/0.6774 | 24.93/0.7827 | 30.12/0.9193 | 33.92/0.9601 | 26.18/0.7639 | |
SPLNet | 19.55/0.4873 | 23.55/0.7301 | 26.19/0.8290 | 32.11/0.9405 | 36.41/0.9737 | 27.56/0.7921 | |
AMP-Net | 19.62/0.4969 | 23.45/0.7290 | 26.04/0.8283 | 32.19/0.9418 | 36.33/0.9737 | 27.53/0.7939 | |
OPINENet+ | 19.38/0.4872 | 23.70/0.7363 | 26.61/0.8362 | 32.58/0.9414 | 36.62/0.9727 | 27.78/0.7948 | |
FSOINet | |||||||
FSOINet+ | 20.14/0.5331 | 24.80/0.7805 | 28.05/0.8718 | 34.29/0.9569 | 38.31/0.9800 | 29.12/0.8245 |
表2 各采样率不同算法在不同数据集上重构图像PSNR(dB)/SSIM对比
数据集 | 方法 | 采样率 | 平均 | ||||
---|---|---|---|---|---|---|---|
0.01 | 0.05 | 0.1 | 0.3 | 0.5 | |||
BSDS68 | CSNet+ | 21.71/0.5249 | 25.04/0.6845 | 26.89/0.7756 | 31.66/0.9152 | 35.42/0.9614 | 28.14/0.7723 |
SCSNet | 21.88/0.5250 | 24.98/0.6843 | 27.13/0.7785 | 31.76/0.9173 | 35.67/0.9640 | 28.28/0.7738 | |
SPLNet | 22.33/0.5242 | 25.87/0.7198 | 27.85/0.8094 | 32.77/0.9303 | 36.86/0.9708 | 29.13/0.7907 | |
AMP-Net | 22.28/0.5315 | 25.77/0.7204 | 27.85/0.8113 | 32.84/0.9321 | 36.82/0.9715 | 29.11/0.7934 | |
OPINENet+ | 21.88/0.5162 | 25.66/0.7136 | 27.81/0.8040 | 32.50/0.9236 | 36.32/0.9658 | 28.83/0.7846 | |
FSOINet | |||||||
FSOINet+ | 22.83/0.5441 | 26.27/0.7340 | 28.39/0.8210 | 33.37/0.9352 | 37.47/0.9732 | 29.67/0.8015 | |
Urban100 | CSNet+ | 19.27/0.4812 | 22.63/0.6792 | 24.64/0.7741 | 29.90/0.9162 | 33.55/0.9572 | 26.00/0.7616 |
SCSNet | 19.28/0.4798 | 22.63/0.6774 | 24.93/0.7827 | 30.12/0.9193 | 33.92/0.9601 | 26.18/0.7639 | |
SPLNet | 19.55/0.4873 | 23.55/0.7301 | 26.19/0.8290 | 32.11/0.9405 | 36.41/0.9737 | 27.56/0.7921 | |
AMP-Net | 19.62/0.4969 | 23.45/0.7290 | 26.04/0.8283 | 32.19/0.9418 | 36.33/0.9737 | 27.53/0.7939 | |
OPINENet+ | 19.38/0.4872 | 23.70/0.7363 | 26.61/0.8362 | 32.58/0.9414 | 36.62/0.9727 | 27.78/0.7948 | |
FSOINet | |||||||
FSOINet+ | 20.14/0.5331 | 24.80/0.7805 | 28.05/0.8718 | 34.29/0.9569 | 38.31/0.9800 | 29.12/0.8245 |
方法 | 参数量 | 0.5采样率 | 0.1采样率 | ||
---|---|---|---|---|---|
Set11 | BSDS68 | Set11 | BSDS68 | ||
平均运行时间(s) | |||||
SPLNet | 1.388M | 0.0061 | 0.0076 | 0.0090 | 0.0089 |
AMPNet | 1.529M | 0.0562 | 0.0671 | 0.0564 | 0.0649 |
OPINENet+ | 1.095M | 0.0087 | 0.0126 | 0.0134 | 0.0132 |
FSOINet | 1.061M | 0.0198 | 0.0184 | 0.0215 | 0.0190 |
FSOINet+ | 1.086M | 0.0294 | 0.0264 | 0.0283 | 0.0258 |
表3 不同采样率不同算法在Set11与BSDS68上的复杂度对比
方法 | 参数量 | 0.5采样率 | 0.1采样率 | ||
---|---|---|---|---|---|
Set11 | BSDS68 | Set11 | BSDS68 | ||
平均运行时间(s) | |||||
SPLNet | 1.388M | 0.0061 | 0.0076 | 0.0090 | 0.0089 |
AMPNet | 1.529M | 0.0562 | 0.0671 | 0.0564 | 0.0649 |
OPINENet+ | 1.095M | 0.0087 | 0.0126 | 0.0134 | 0.0132 |
FSOINet | 1.061M | 0.0198 | 0.0184 | 0.0215 | 0.0190 |
FSOINet+ | 1.086M | 0.0294 | 0.0264 | 0.0283 | 0.0258 |
模型设置 | 测试集PSNR(dB) | ||||
---|---|---|---|---|---|
FSIM | DDM | WSEM | Set11 | BSDS68 | Urban100 |
√ | × | × | 40.22 | 36.82 | 36.31 |
× | √ | × | 39.65 | 36.36 | 35.38 |
√ | √ | × | 41.10 | 37.36 | 37.91 |
√ | √ | √ | 41.29 | 37.47 | 38.31 |
表4 0.5采样率下不同模块的消融实验
模型设置 | 测试集PSNR(dB) | ||||
---|---|---|---|---|---|
FSIM | DDM | WSEM | Set11 | BSDS68 | Urban100 |
√ | × | × | 40.22 | 36.82 | 36.31 |
× | √ | × | 39.65 | 36.36 | 35.38 |
√ | √ | × | 41.10 | 37.36 | 37.91 |
√ | √ | √ | 41.29 | 37.47 | 38.31 |
训练策略 | 0.01采样率 | 0.1采样率 | 0.3采样率 |
---|---|---|---|
随机初始化训练40 epoches | 21.77/0.5920 | 30.31/0.8995 | 36.74/0.9651 |
随机初始化训练100 epoches | 21.81/0.5940 | 30.51/0.9021 | 36.96/0.9660 |
迁移学习40 epoches | 21.88/0.5958 | 30.57/0.9020 | 37.00/0.9664 |
表5 不同训练策略的FSOINet在Set11上的重构图像PSNR(dB)/SSIM
训练策略 | 0.01采样率 | 0.1采样率 | 0.3采样率 |
---|---|---|---|
随机初始化训练40 epoches | 21.77/0.5920 | 30.31/0.8995 | 36.74/0.9651 |
随机初始化训练100 epoches | 21.81/0.5940 | 30.51/0.9021 | 36.96/0.9660 |
迁移学习40 epoches | 21.88/0.5958 | 30.57/0.9020 | 37.00/0.9664 |
1 | DONOHO D L. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306. |
2 | DUARTE M F, DAVENPORT M A, TAKHAR D, et al. Single-pixel imaging via compressive sampling[J]. IEEE signal processing magazine, 2008, 25(2): 83-91. |
3 | ZHU L, WU X, SUN Z, et al. Compressed-sensing accelerated 3-dimensional magnetic resonance cholangiopancreatography: application in suspected pancreatic diseases[J]. Investigative radiology, 2018, 53(3): 150-157. |
4 | YUAN X, BRADY D J, KATSAGGELOS A K. Snapshot compressive imaging: theory, algorithms, and applications[J]. IEEE Signal Processing Magazine, 2021, 38(2): 65-88. |
5 | 禤韵怡, 杨春玲. 基于帧间组稀疏的两阶段递归增强视频压缩感知重构网络[J]. 电子学报, 2021, 49(03): 435-442. |
XUAN Yun-yi, YANG Chun-ling. Two-stage recursive enhancement reconstruction based on video inter-frame group sparse representation in compressed video sensing[J]. Acta Electronica Sinica, 2021, 49(3): 435-442. (in Chinese) | |
6 | GAN L. Block compressed sensing of natural images[C]//15th International Conference on Digital Signal Processing. Cardiff, UK: IEEE, 2007: 403-406. |
7 | CHEN C, TRAMEL E W, FOWLER J E. Compressed-sensing recovery of images and video using multihypothesis predictions[C]//2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers. Pacific Grove, USA: IEEE, 2011: 1193-1198. |
8 | ZHANG J, ZHAO D, GAO W. Group-based sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2014, 23(8): 3336-3351. |
9 | KULKARNI K, LOHIT S, TURAGA P, et al. Reconnet: non-iterative reconstruction of images from compressively sensed measurements[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 449-458. |
10 | SHI W, JIANG F, ZHANG S, et al. Deep networks for compressed image sensing[C]//2017 IEEE International Conference on Multimedia and Expo. Hong Kong, China: IEEE, 2017: 877-882. |
11 | SHI W, JIANG F, LIU S, et al. Scalable convolutional neural network for image compressed sensing[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 12290-12299. |
12 | SHI W, JIANG F, LIU S, et al. Image compressed sensing using convolutional neural network[J]. IEEE Transactions on Image Processing, 2020, 29: 375-388. |
13 | CHEN J, SUN Y, LIU Q, et al. Learning memory augmented cascading network for compressed sensing of images[C]//2020 European Conference on Computer Vision. Glasgow, UK: Springer, 2020: 513-529. |
14 | ZHOU S, HE Y, LIU Y, et al. Multi-channel deep networks for block-based image compressive sensing[J]. IEEE Transactions on Multimedia, 2021, 23: 2627-2640. |
15 | ZHANG J, GHANEM B. ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 1828-1837. |
16 | 裴翰奇, 杨春玲, 魏志超, 曹燕. 基于SPL迭代思想的图像压缩感知重构神经网络[J]. 电子学报, 2021, 49(6): 1195-1203. |
PEI Han-qi, YANG Chun-ling, WEI Zhi-chao, CAO Yan. Image compressive sensing reconstruction network based on iterative SPL theory[J]. Acta Electronica Sinica, 2021, 49(6): 1195-1203. (in Chinese) | |
17 | ZHANG Z, LIU Y, LIU J, et al. AMP-Net: denoising-based deep unfolding for compressive image sensing[J]. IEEE Transactions on Image Processing, 2021, 30: 1487-1500. |
18 | ZHANG J, ZHAO C, GAO W. Optimization-inspired compact deep compressive sensing[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(4): 765-774. |
19 | SONG J, CHEN B, ZHANG J. Memory-augmented deep unfolding network for compressive sensing[C]//2021 ACM International Conference on Multimedia. Chengdu, China: ACM, 2021: 4249-4258. |
20 | D'ASCOLI S, Touvron H, Leavitt M L, et al. Convit: improving vision transformers with soft convolutional inductive biases[C]//2021 International Conference on Machine Learning. Virtual Only: ACM, 2021: 2286-2296. |
21 | ARBEL´AEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 898-916. |
22 | KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2022-04-11]. . |
23 | LOSHCHILOV I, HUTTER F. Sgdr: Stochastic gradient descent with warm restarts[EB/OL]. [2022-04-11]. . |
24 | MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//2001 IEEE International Conference on Computer Vision. Vancouver, Canada: IEEE, 2001: 2:416-423. |
25 | HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 5197-5206. |
26 | CHEN W, YANG C, YANG X. FSOINET: feature-space optimization-inspired network for image compressive sensing[EB/OL]. [2022-04-11]. . |
[1] | 潘敏婷, 王韫博, 朱祥明, 高思宇, 龙明盛, 杨小康. 基于无标签视频数据的深度预测学习方法综述[J]. 电子学报, 2022, 50(4): 869-886. |
[2] | 杨曦, 张鑫, 郭浩远, 王楠楠, 高新波. 基于不变特征的多源遥感图像舰船目标检测算法[J]. 电子学报, 2022, 50(4): 887-899. |
[3] | 张浩, 胡昌华, 杜党波, 裴洪, 张建勋. 多状态影响下基于Bi‑LSTM网络的锂电池剩余寿命预测方法[J]. 电子学报, 2022, 50(3): 619-624. |
[4] | 崔亚奇, 何友, 唐田田, 熊伟. 一种深度学习航迹关联方法[J]. 电子学报, 2022, 50(3): 759-763. |
[5] | 李维刚, 甘平, 谢璐, 李松涛. 基于样本对元学习的小样本图像分类方法[J]. 电子学报, 2022, 50(2): 295-304. |
[6] | 骈纬国, 吴映波, 陈蒙, 蔡俊鹏. 一种基于时空动态图注意力网络的共享出行需求预测方法[J]. 电子学报, 2022, 50(2): 432-439. |
[7] | 伍邦谷, 张苏林, 石红, 朱鹏飞, 王旗龙, 胡清华. 基于多分支结构的不确定性局部通道注意力机制[J]. 电子学报, 2022, 50(2): 374-382. |
[8] | 张小恒, 张馨月, 李勇明, 王品, 刘玉川. 面向帕金森病语音诊断的非监督两步式卷积稀疏迁移学习算法[J]. 电子学报, 2022, 50(1): 177-184. |
[9] | 李居朋, 王颖慧, 李刚. 医学图像关键点检测深度学习方法研究与挑战[J]. 电子学报, 2022, 50(1): 226-237. |
[10] | 李勇斌, 王彬, 邵高平, 邵帅. 一种基于RCGAN的水声通信信号降噪方法[J]. 电子学报, 2022, 50(1): 54-62. |
[11] | 张聿远, 张立民, 闫文君. 基于深度多级残差网络的低信噪比下空频分组码识别方法[J]. 电子学报, 2022, 50(1): 79-88. |
[12] | 赵琰, 赵凌君, 匡纲要. 基于注意力机制特征融合网络的SAR图像飞机目标快速检测[J]. 电子学报, 2021, 49(9): 1665-1674. |
[13] | 管永明, 王刚, 骆凯波, 吕梁, 吕晓雯, 史玉良. 基于极端不平衡学习的泛化低压异常箱表关系识别研究与应用[J]. 电子学报, 2021, 49(8): 1507-1514. |
[14] | 席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265. |
[15] | 谢佳龙, 张波涛, 吕强. 一种基于双流融合3D卷积神经网络的动态头势识别方法[J]. 电子学报, 2021, 49(7): 1363-1369. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||