基于深监督跨尺度注意力网络的深度图像超分辨率重建

doi:10.12263/DZXB.20210659

电子学报 ›› 2023, Vol. 51 ›› Issue (1): 128-138.DOI: 10.12263/DZXB.20210659

基于深监督跨尺度注意力网络的深度图像超分辨率重建

李滔¹, 董秀成¹, 林宏伟²

^1.西华大学电气与电子信息学院，四川成都 610039
^2.西北民族大学电气工程学院，甘肃兰州 730000

收稿日期:2021-05-24 修回日期:2022-05-17 出版日期:2023-01-25
- 作者简介:
- 李　滔　女，1983年8月出生，四川资阳人.分别于2005年、2008年和2017年在四川大学获得学士、硕士和博士学位.现为西华大学副教授、硕士生导师.主要从事数字图像处理及计算机视觉方面的研究工作.E-mail: lucia634@163.com
  董秀成男，1963年4月出生，陕西咸阳人.分别于1985年和1990年在重庆大学获得学士和硕士学位.现为西华大学教授、硕士生导师.主要从事现代控制理论及机器人方面的研究工作.
  林宏伟　男，1983年2月出生.于2019年在四川大学获得博士学位.现为西北民族大学副教授、硕士生导师.主要从事数字图像处理、视频压缩及通信方面的研究工作.
- 基金资助:
- 国家自然科学基金 (61901392); 四川省科技计划 (2021YJ0109)

Depth Map Super-Resolution Reconstruction Based on Deeply Supervised Cross-Scale Attention Network

LI Tao¹, DONG Xiu-cheng¹, LIN Hong-wei²

^1.School of Electrical Engineering and Electronic Information, Xihua University, Chengdu, Sichuan 610039, China
^2.College of Electrical Engineering, Northwest Minzu University, Lanzhou, Gansu 730000, China

Received:2021-05-24 Revised:2022-05-17 Online:2023-01-25 Published:2023-02-23
- Supported by:
- National Natural Science Foundation of China (61901392); Department of Science and Technology of Sichuan Province (2021YJ0109)

摘要/Abstract

摘要：

消费级深度相机拍摄的深度图像具有分辨率较低的问题，深度图像超分辨率重建是解决该问题的有效方法.为了提高重建性能，提出一种基于深监督跨尺度注意力网络的深度图像超分辨率重建算法.网络逐级放大，在损失函数中对每一级的输出都进行约束，实现深监督的目的.采用高阶跨尺度注意力模块，将多尺度特征尺度内及跨尺度相关性与注意力机制结合起来，实现多尺度特征的自适应调整.采用内层为宽激活残差、外层为基本残差的双层残差块作为网络基本构成元素，以提高网络对复杂非线性关系的学习能力.实验结果表明，本文算法在主观视觉效果和客观质量评价指标方面都优于当前主流的深度图像超分辨率重建算法.

关键词: 深度图像超分辨率, 深度学习, 深监督, 多尺度特征表示, 残差块

Abstract:

Depth maps captured by consumer depth cameras usually suffer from low spatial resolution. Depth map super-resolution (SR) is an effective method to solve this problem. To improve the reconstruction performance, this paper proposes a depth map super-resolution reconstruction algorithm based on deeply supervised cross-scale attention network. A multi-stage up-sampling strategy is introduced. The loss function of the network contains the constraint on the output of each stage for a deep supervision. A high-order cross-scale attention block is proposed to adaptively adjust multi-scale features by integrating the in-scale and cross-scale correlations of multi-scale features with the attention mechanism. A bilayer residual block, which contains inner wide-activated residual learning and outer basic residual learning, is used as the basic component of network for more powerful ability of complex non-linear relationship learning. Experimental results demonstrate the superiority of the proposed algorithm over several state-of-the-art depth map SR methods in terms of visual comparison and quantitative evaluation.

Key words: depth map super-resolution, deep learning, deep supervision, multi-scale feature representation, residual block

中图分类号:

TP751.1

李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.

Tao LI, Xiu-cheng DONG, Hong-wei LIN . Depth Map Super-Resolution Reconstruction Based on Deeply Supervised Cross-Scale Attention Network[J]. Acta Electronica Sinica, 2023, 51(1): 128-138.

图/表 12

参考文献 33

1	HORNACEK M, RHEMANN C, GELAUTZ M, et al. Depth super resolution by rigid body self-similarity in 3d[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 1123-1130.
2	LEI J, LI L, YUE H, et al. Depth map super- resolution considering view synthesis quality[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1732-1745.
3	MAC A O, CAMPBELL N D F, NAIR A, et al. Patch based synthesis for single depth image super-resolution[C]//European Conference on Computer Vision. Berlin: Springer, 2012: 71-84.
4	XIE J, FERIS R S, SUN M T. Edge-guided single depth image super resolution[J]. IEEE Transactions on Image Processing, 2015, 25(1): 428-438.
5	XIE J, FERIS R S, YU S S, et al. Joint super resolution and denoising from a single depth image[J]. IEEE Transactions on Multimedia, 2015, 17(9): 1525-1537.
6	MANDAL S, BHAVSAR A, SAO A K. Depth map restoration from undersampled data[J]. IEEE Transactions on Image Processing, 2016, 26(1): 119-134.
7	RIEGLER G, et al. Atgv-net: Accurate depth super-resolution[C]//European Conference on Computer Vision. Cham: Springer, 2016: 268-284.
8	SONG X, DAI Y, QIN X. Deeply supervised depth map super-resolution as novel view synthesis[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(8): 2323-2336.
9	HUANG L, ZHANG J, ZUO Y, et al. Pyramid-structured depth MAP super-resolution based on deep dense-residual network[J]. IEEE Signal Processing Letters, 2019, 26(12): 1723-1727.
10	SONG X, DAI Y, ZHOU D, et al. Channel attention based iterative residual learning for depth map super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5631-5640.
11	HUI T W, LOY C C, TANG X. Depth map super-resolution by deep multi-scale guidance[C]//European Conference on Computer Vision. Cham: Springer, 2016: 353-369.
12	KIECHLE M, HAWE S, KLEINSTEUBER M. A joint intensity and depth cosparse analysis model for depth map super-resolution[C]//IEEE International Conference on Computer Vision. Piscataway: IEEE, 2013: 1545-1552.
13	GUO C, LI C, GUO J, et al. Hierarchical features driven residual learning for depth map super-resolution[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2545-2557.
14	ZUO Y, WU Q, FANG Y, et al. Multi-scale frequency reconstruction for guided depth map super-resolution via deep residual network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(2): 297-306.
15	LI Y, HUANG J B, AHUJA N, YANG M H. Joint image filtering with deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8): 1909-1923.
16	LI T, LIN H, DONG X, et al. Depth image super-resolution using correlation-controlled color guidance and multi-scale symmetric network[J]. Pattern Recognition, 2020, 107: 107513.
17	LI T, DONG X, LIN H. Guided depth map super-resolution using recumbent Y network[J]. IEEE Access, 2020, 8: 122695-122708.
18	ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision. Munich: Springer, 2018: 286-301.
19	DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 11065-11074.
20	HU Y, LI J, HUANG Y, et al. Channel-wise and spatial feature modulation network for single image super-resolution[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(11): 3911-3927.
21	ZHANG Y, LI K, LI K, et al. Residual non-local attention networks for image restoration[C]//International Conference on Learning Representations. New Orleans: Brown Walker Press, 2019.
22	MEI Y, FAN Y, ZHOU Y, et al. Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5690-5699.
23	NAJIBI M, SAMANGOUEI P, CHELLAPPA R, et al. SSH: Single stage headless face detector[C]//IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 4875-4884.
24	LIU Y, CHENG M M, HU X, et al. Richer convolutional features for edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(8): 1939-1946.
25	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Munich: Springer, 2018: 801-818.
26	李雅倩, 盖成远, 肖存军, 等. 基于细化多尺度深度特征的目标检测网络[J]. 电子学报, 2020, 48(12): 2360-2366.
	LI Y, GAI C, XIAO C, et al. Object detection networks based on refined multi-scale depth feature [J]. Acta Electronica Sinica, 2020, 48(12): 2360-2366. (in Chinese)
27	ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2881-2890.
28	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848.
29	SHI W, CABALLERO J, HUSZAR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1874-1883.
30	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2016: 770-778.
31	FAN Y, YU J, HUANG T S. Wide-activated deep residual networks based restoration for BPG-compressed images[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2621-2624.
32	JIN T, HUANG S, LI Y, et al. Low-rank HOCA: Efficient high-order cross-modal attention for video captioning[C]//2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong: ACL, 2019: 2001-2011.
33	Middlebury. The Middlebury Stereo Datasets[DB/OL]. (2015). .

方法	Art		Book		Dolls		Laundry		Moebius		Reindeer		Cones		Teddy		Tsukuba		Venus		平均
方法	PSNR	PE	PSNR	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE
2倍
Bicubic	39.74	2.750	47.77	0.687	48.96	0.709	44.03	1.305	49.32	0.754	42.45	1.309	40.14	3.649	42.40	2.613	32.82	6.884	45.84	1.044	43.35	2.170
RCAN	55.35	0.084	57.11	0.023	55.01	0.070	56.10	0.058	55.99	0.073	55.94	0.048	51.31	0.271	51.25	0.284	51.58	0.269	55.92	0.010	54.56	0.119
SAN	55.10	0.094	57.19	0.023	55.00	0.073	56.07	0.061	55.89	0.074	55.90	0.050	51.19	0.254	51.09	0.299	53.66	0.253	55.32	0.014	54.64	0.120
ATGV-Net	54.11	0.108	60.29	0.023	57.04	0.054	56.10	0.073	58.46	0.055	55.59	0.061	46.77	0.643	49.97	0.482	39.65	1.488	46.41	2.336	52.44	0.532
JID	46.06	0.685	51.62	0.262	50.58	0.355	50.36	0.398	52.30	0.301	47.55	0.411	44.21	1.500	46.56	1.165	37.22	2.386	52.19	0.249	47.87	0.771
MSG-Net	53.16	0.148	59.94	0.025	56.70	0.043	56.75	0.061	58.17	0.059	55.59	0.069	49.00	0.611	51.07	0.384	42.80	1.247	65.08	0.006	54.83	0.265
DepthSR-Net	53.64	0.050	55.54	0.011	54.27	0.024	55.23	0.032	55.29	0.029	53.86	0.033	50.15	0.171	49.78	0.276	45.46	0.336	54.06	0.001	52.73	0.096
RYNet	59.88	0.040	63.09	0.008	59.45	0.026	61.23	0.027	60.66	0.028	60.12	0.026	54.10	0.175	52.84	0.269	47.49	0.312	65.58	0.001	58.44	0.091
Proposed	60.33	0.037	63.05	0.008	58.41	0.025	61.02	0.026	60.73	0.029	60.14	0.025	54.02	0.167	52.91	0.255	51.16	0.233	64.94	0.001	58.67	0.081
4倍
Bicubic	36.38	5.507	44.05	1.465	45.80	1.488	40.52	2.755	45.63	1.658	39.19	2.546	36.45	7.207	39.04	5.276	29.47	13.052	42.52	2.200	39.91	4.32
RCAN	45.19	0.470	52.80	0.125	49.35	0.315	49.45	0.308	50.93	0.295	48.00	0.265	41.61	1.690	44.09	2.204	37.15	2.031	51.28	0.138	46.99	0.78
SAN	45.06	0.565	52.90	0.128	49.70	0.310	49.47	0.317	51.25	0.288	47.91	0.291	41.57	1.804	44.38	1.939	36.79	2.236	51.06	0.175	47.01	0.81
ATGV-Net	44.52	0.542	54.63	0.084	50.30	0.238	48.61	0.405	52.26	0.211	47.56	0.244	38.84	2.670	44.70	1.761	31.65	4.697	58.05	0.041	47.11	1.09
DJFR	36.73	3.109	44.27	0.779	45.38	1.019	40.99	1.601	46.07	0.978	39.91	1.392	37.11	4.570	39.22	3.540	29.68	8.252	43.32	0.993	40.27	2.62
JID	42.29	1.238	49.73	0.417	48.56	0.610	45.85	0.867	49.51	0.576	44.93	0.690	39.03	3.157	43.18	2.508	32.66	4.699	49.38	0.427	44.51	1.52
MSG-Net	45.23	0.550	54.94	0.089	50.85	0.206	50.21	0.256	52.86	0.192	48.27	0.260	39.85	2.277	44.69	1.581	35.48	4.350	57.34	0.046	47.97	0.98
DepthSR-Net	46.42	0.224	52.35	0.056	49.97	0.152	50.22	0.161	51.54	0.157	48.40	0.137	40.77	1.249	45.37	1.129	37.78	1.171	52.01	0.014	47.48	0.45
RYNet	48.29	0.236	57.11	0.049	52.66	0.160	52.06	0.161	54.18	0.152	50.70	0.138	42.46	1.193	45.48	1.819	37.73	1.286	60.28	0.019	50.10	0.52
Proposed	48.88	0.219	57.13	0.047	52.37	0.150	52.96	0.138	54.30	0.146	51.63	0.136	45.17	0.935	45.72	1.966	40.01	1.115	60.03	0.014	50.82	0.49
8倍
Bicubic	33.39	10.534	40.79	3.098	42.75	3.075	37.38	5.600	42.23	3.610	36.14	4.967	33.29	12.932	35.99	10.268	26.31	20.858	39.35	4.454	36.76	7.94
RCAN	38.67	2.476	46.59	0.681	45.86	0.955	42.98	1.366	46.88	0.912	41.40	1.189	35.38	5.823	38.77	5.798	30.30	8.041	44.61	0.941	41.14	2.82
SAN	39.21	1.637	48.26	0.425	45.71	0.841	44.23	1.008	47.08	0.752	42.36	0.784	35.58	5.010	39.01	5.377	30.59	6.432	45.31	0.574	41.73	2.28
ATGV-Net	38.15	2.829	47.50	0.630	45.65	0.990	42.71	2.139	47.18	0.850	41.86	1.092	33.90	7.339	39.01	5.520	26.87	12.184	47.27	0.406	41.01	3.40
DJFR	34.43	5.463	41.48	1.544	43.27	1.976	38.45	2.928	42.54	2.132	37.12	2.559	32.39	7.674	37.27	6.077	25.11	14.445	41.22	1.843	37.33	4.66
JID	39.27	1.794	47.81	0.602	46.16	0.992	42.74	1.483	46.61	1.003	41.16	1.185	35.04	5.403	39.53	4.984	28.42	8.064	46.35	0.622	41.31	2.61
MSG-Net	40.45	1.411	49.24	0.356	47.34	0.609	44.53	1.044	48.70	0.508	43.24	0.744	35.61	5.625	39.31	4.550	29.62	10.059	47.79	0.294	42.58	2.52
DepthSR-Net	41.02	0.892	49.03	0.212	47.21	0.469	45.81	0.648	48.47	0.417	44.23	0.420	37.36	3.797	40.16	4.105	31.30	5.990	47.26	0.194	43.19	1.71
RYNet	41.94	0.795	50.88	0.215	48.40	0.441	46.47	0.581	49.96	0.388	45.13	0.354	37.68	3.226	40.97	3.665	31.44	4.899	51.26	0.142	44.41	1.48
Proposed	42.58	0.696	52.09	0.166	48.11	0.455	47.55	0.495	50.27	0.358	45.30	0.351	37.09	3.729	40.03	4.606	32.22	3.846	52.82	0.113	44.81	1.48

方法	Art		Book		Dolls		Laundry		Moebius		Reindeer		Cones		Teddy		Tsukuba		Venus		平均
方法	PSNR	PE	PSNR	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE
2倍
Bicubic	39.74	2.750	47.77	0.687	48.96	0.709	44.03	1.305	49.32	0.754	42.45	1.309	40.14	3.649	42.40	2.613	32.82	6.884	45.84	1.044	43.35	2.170
RCAN	55.35	0.084	57.11	0.023	55.01	0.070	56.10	0.058	55.99	0.073	55.94	0.048	51.31	0.271	51.25	0.284	51.58	0.269	55.92	0.010	54.56	0.119
SAN	55.10	0.094	57.19	0.023	55.00	0.073	56.07	0.061	55.89	0.074	55.90	0.050	51.19	0.254	51.09	0.299	53.66	0.253	55.32	0.014	54.64	0.120
ATGV-Net	54.11	0.108	60.29	0.023	57.04	0.054	56.10	0.073	58.46	0.055	55.59	0.061	46.77	0.643	49.97	0.482	39.65	1.488	46.41	2.336	52.44	0.532
JID	46.06	0.685	51.62	0.262	50.58	0.355	50.36	0.398	52.30	0.301	47.55	0.411	44.21	1.500	46.56	1.165	37.22	2.386	52.19	0.249	47.87	0.771
MSG-Net	53.16	0.148	59.94	0.025	56.70	0.043	56.75	0.061	58.17	0.059	55.59	0.069	49.00	0.611	51.07	0.384	42.80	1.247	65.08	0.006	54.83	0.265
DepthSR-Net	53.64	0.050	55.54	0.011	54.27	0.024	55.23	0.032	55.29	0.029	53.86	0.033	50.15	0.171	49.78	0.276	45.46	0.336	54.06	0.001	52.73	0.096
RYNet	59.88	0.040	63.09	0.008	59.45	0.026	61.23	0.027	60.66	0.028	60.12	0.026	54.10	0.175	52.84	0.269	47.49	0.312	65.58	0.001	58.44	0.091
Proposed	60.33	0.037	63.05	0.008	58.41	0.025	61.02	0.026	60.73	0.029	60.14	0.025	54.02	0.167	52.91	0.255	51.16	0.233	64.94	0.001	58.67	0.081
4倍
Bicubic	36.38	5.507	44.05	1.465	45.80	1.488	40.52	2.755	45.63	1.658	39.19	2.546	36.45	7.207	39.04	5.276	29.47	13.052	42.52	2.200	39.91	4.32
RCAN	45.19	0.470	52.80	0.125	49.35	0.315	49.45	0.308	50.93	0.295	48.00	0.265	41.61	1.690	44.09	2.204	37.15	2.031	51.28	0.138	46.99	0.78
SAN	45.06	0.565	52.90	0.128	49.70	0.310	49.47	0.317	51.25	0.288	47.91	0.291	41.57	1.804	44.38	1.939	36.79	2.236	51.06	0.175	47.01	0.81
ATGV-Net	44.52	0.542	54.63	0.084	50.30	0.238	48.61	0.405	52.26	0.211	47.56	0.244	38.84	2.670	44.70	1.761	31.65	4.697	58.05	0.041	47.11	1.09
DJFR	36.73	3.109	44.27	0.779	45.38	1.019	40.99	1.601	46.07	0.978	39.91	1.392	37.11	4.570	39.22	3.540	29.68	8.252	43.32	0.993	40.27	2.62
JID	42.29	1.238	49.73	0.417	48.56	0.610	45.85	0.867	49.51	0.576	44.93	0.690	39.03	3.157	43.18	2.508	32.66	4.699	49.38	0.427	44.51	1.52
MSG-Net	45.23	0.550	54.94	0.089	50.85	0.206	50.21	0.256	52.86	0.192	48.27	0.260	39.85	2.277	44.69	1.581	35.48	4.350	57.34	0.046	47.97	0.98
DepthSR-Net	46.42	0.224	52.35	0.056	49.97	0.152	50.22	0.161	51.54	0.157	48.40	0.137	40.77	1.249	45.37	1.129	37.78	1.171	52.01	0.014	47.48	0.45
RYNet	48.29	0.236	57.11	0.049	52.66	0.160	52.06	0.161	54.18	0.152	50.70	0.138	42.46	1.193	45.48	1.819	37.73	1.286	60.28	0.019	50.10	0.52
Proposed	48.88	0.219	57.13	0.047	52.37	0.150	52.96	0.138	54.30	0.146	51.63	0.136	45.17	0.935	45.72	1.966	40.01	1.115	60.03	0.014	50.82	0.49
8倍
Bicubic	33.39	10.534	40.79	3.098	42.75	3.075	37.38	5.600	42.23	3.610	36.14	4.967	33.29	12.932	35.99	10.268	26.31	20.858	39.35	4.454	36.76	7.94
RCAN	38.67	2.476	46.59	0.681	45.86	0.955	42.98	1.366	46.88	0.912	41.40	1.189	35.38	5.823	38.77	5.798	30.30	8.041	44.61	0.941	41.14	2.82
SAN	39.21	1.637	48.26	0.425	45.71	0.841	44.23	1.008	47.08	0.752	42.36	0.784	35.58	5.010	39.01	5.377	30.59	6.432	45.31	0.574	41.73	2.28
ATGV-Net	38.15	2.829	47.50	0.630	45.65	0.990	42.71	2.139	47.18	0.850	41.86	1.092	33.90	7.339	39.01	5.520	26.87	12.184	47.27	0.406	41.01	3.40
DJFR	34.43	5.463	41.48	1.544	43.27	1.976	38.45	2.928	42.54	2.132	37.12	2.559	32.39	7.674	37.27	6.077	25.11	14.445	41.22	1.843	37.33	4.66
JID	39.27	1.794	47.81	0.602	46.16	0.992	42.74	1.483	46.61	1.003	41.16	1.185	35.04	5.403	39.53	4.984	28.42	8.064	46.35	0.622	41.31	2.61
MSG-Net	40.45	1.411	49.24	0.356	47.34	0.609	44.53	1.044	48.70	0.508	43.24	0.744	35.61	5.625	39.31	4.550	29.62	10.059	47.79	0.294	42.58	2.52
DepthSR-Net	41.02	0.892	49.03	0.212	47.21	0.469	45.81	0.648	48.47	0.417	44.23	0.420	37.36	3.797	40.16	4.105	31.30	5.990	47.26	0.194	43.19	1.71
RYNet	41.94	0.795	50.88	0.215	48.40	0.441	46.47	0.581	49.96	0.388	45.13	0.354	37.68	3.226	40.97	3.665	31.44	4.899	51.26	0.142	44.41	1.48
Proposed	42.58	0.696	52.09	0.166	48.11	0.455	47.55	0.495	50.27	0.358	45.30	0.351	37.09	3.729	40.03	4.606	32.22	3.846	52.82	0.113	44.81	1.48

方法	Art		Book		Dolls		Laundry		Moebius		Reindeer		Cones		Teddy		Tsukuba		Venus		平均
方法	PSNR	PE	PSNR	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE
Bicubic	31.42	9.075	34.57	3.995	34.96	3.813	33.52	5.093	34.92	3.896	32.93	5.264	31.36	9.493	32.97	7.455	25.85	15.353	34.01	5.336	32.65	6.877
RCAN	35.75	2.229	42.54	0.540	40.55	0.954	39.12	1.424	41.55	0.906	38.71	1.015	32.40	5.565	35.94	3.718	27.99	8.457	41.12	0.803	37.57	2.561
SAN	35.67	2.067	42.60	0.542	40.90	0.912	38.94	1.329	41.67	0.875	38.38	0.944	31.94	5.642	35.91	3.514	28.15	7.978	41.59	0.742	37.58	2.455
ATGV-Net	35.01	2.876	40.70	0.714	40.19	0.875	37.96	1.199	40.35	0.960	37.84	1.145	32.78	5.540	35.84	3.810	26.70	9.325	39.57	0.979	36.69	2.742
MSG-Net	35.46	2.393	40.89	0.584	40.24	0.819	38.29	1.091	39.84	1.000	37.89	1.281	32.88	5.468	36.02	3.389	27.15	9.580	40.02	0.750	36.87	2.636
DepthSR-Net	36.17	1.611	41.77	0.420	41.07	0.623	39.33	0.722	41.35	0.687	38.72	0.733	33.53	4.407	36.43	2.847	27.50	7.502	41.58	0.366	37.75	1.992
RYNet	37.33	1.197	42.66	0.351	42.23	0.465	40.29	0.633	42.67	0.531	39.55	0.620	34.61	3.455	37.37	2.403	28.34	6.264	41.37	0.330	38.64	1.625
Proposed	36.22	1.998	42.21	0.535	40.83	0.812	39.20	0.984	41.46	0.847	38.68	0.916	32.99	4.997	36.27	3.261	28.80	7.489	40.58	0.810	37.72	2.265

方法	Art		Book		Dolls		Laundry		Moebius		Reindeer		Cones		Teddy		Tsukuba		Venus		平均
方法	PSNR	PE	PSNR	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE	PSRN	PE
Bicubic	31.42	9.075	34.57	3.995	34.96	3.813	33.52	5.093	34.92	3.896	32.93	5.264	31.36	9.493	32.97	7.455	25.85	15.353	34.01	5.336	32.65	6.877
RCAN	35.75	2.229	42.54	0.540	40.55	0.954	39.12	1.424	41.55	0.906	38.71	1.015	32.40	5.565	35.94	3.718	27.99	8.457	41.12	0.803	37.57	2.561
SAN	35.67	2.067	42.60	0.542	40.90	0.912	38.94	1.329	41.67	0.875	38.38	0.944	31.94	5.642	35.91	3.514	28.15	7.978	41.59	0.742	37.58	2.455
ATGV-Net	35.01	2.876	40.70	0.714	40.19	0.875	37.96	1.199	40.35	0.960	37.84	1.145	32.78	5.540	35.84	3.810	26.70	9.325	39.57	0.979	36.69	2.742
MSG-Net	35.46	2.393	40.89	0.584	40.24	0.819	38.29	1.091	39.84	1.000	37.89	1.281	32.88	5.468	36.02	3.389	27.15	9.580	40.02	0.750	36.87	2.636
DepthSR-Net	36.17	1.611	41.77	0.420	41.07	0.623	39.33	0.722	41.35	0.687	38.72	0.733	33.53	4.407	36.43	2.847	27.50	7.502	41.58	0.366	37.75	1.992
RYNet	37.33	1.197	42.66	0.351	42.23	0.465	40.29	0.633	42.67	0.531	39.55	0.620	34.61	3.455	37.37	2.403	28.34	6.264	41.37	0.330	38.64	1.625
Proposed	36.22	1.998	42.21	0.535	40.83	0.812	39.20	0.984	41.46	0.847	38.68	0.916	32.99	4.997	36.27	3.261	28.80	7.489	40.58	0.810	37.72	2.265

方法	Middlebury		ToFMark		NYU_Depth		平均
方法	PSNR	PE	PSNR	PE	PSRN	PE	PSRN	PE
Bicubic	37.27	3.516	40.00	3.204	41.44	2.643	41.27	2.68
RCAN	47.34	0.575	45.43	0.656	45.84	1.023	45.90	1.00
SAN	47.39	0.606	45.58	0.694	46.10	0.986	46.15	0.97
ATGV-Net	47.34	0.787	45.23	0.677	46.31	1.050	46.35	1.04
MSG-Net	48.07	0.691	45.47	0.735	46.18	1.018	46.25	1.00
DepthSR-Net	47.84	0.369	45.59	0.545	45.88	0.830	45.96	0.81
RYNet	50.34	0.354	46.75	0.554	48.02	0.694	48.10	0.68
Proposed	50.89	0.342	47.15	0.525	47.73	0.772	47.85	0.75

基于深监督跨尺度注意力网络的深度图像超分辨率重建

Depth Map Super-Resolution Reconstruction Based on Deeply Supervised Cross-Scale Attention Network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 33

相关文章 15

编辑推荐

Metrics

结构	C1	C2	C3	C4	C5
高阶跨尺度注意力模块	×	√	×	√	√
双层残差块	×	×	√	√	√
深监督	×	×	×	×	√
平均PSNR	41.14	43.15	43.37	43.73	44.81
平均PE	2.82	2.06	1.99	1.80	1.48

方法	RCAN	SAN	ATGV-Net	DJFR	JID
时间	0.45	4.99	0.39	5.62	869.60
显存开销	16	15.7	—	0.5	—
方法	MSG-Net	DepthSR-Net	RYNet	Proposed	—
时间	0.37	1.87	0.71	1.17	—
显存开销	0.8	43.3	62.7	20.5	—

[1]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[2]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[3]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[4]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[5]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[6]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[7]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.
[8]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[9]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[10]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[11]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[12]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[13]	琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.
[14]	张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.
[15]	张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893.