基于穿插特征更新的光场图像超分网络

陈佳妮; 武迎春; 吕天琪; 刘丽; 赵贤凌

doi:10.12263/DZXB.20240253

您当前的位置：

首页 >

文章列表页 >

基于穿插特征更新的光场图像超分网络

学术论文 | 更新时间：2025-12-11

- 基于穿插特征更新的光场图像超分网络
- Cross Feature Updating-Based Network for Light Field Image Super Resolution
- 电子学报 2024年52卷第12期页码：4113-4124
- 作者机构：
  
  太原科技大学电子信息工程学院，山西太原 030024
- 作者简介：
  
  [ "武迎春女，1984年1月出生，山西朔州人.太原科技大学教授，硕士生导师.主要研究方向为3D数据获取与处理、光场信息获取与处理. E-mail: yingchunwu3030@tyust.edu.cn" ]
  [ "吕天琪女，1996年3月出生，河北保定人.太原科技大学硕士研究生.主要研究方向为光场信息获取与处理. E-mail: 2502632776@qq.com" ]
  [ "刘丽女，1978年10月出生，宁夏吴忠人.太原科技大学副教授，硕士生导师.主要研究方向为多媒体数据安全、信息隐藏.中国电子学会会员编号：E190161621M. E-mail: liliu@tyust.edu.cn" ]
  [ "赵贤凌女，1978年12月出生，山西长治人.太原科技大学副教授.主要研究方向为3D数据获取与处理. E-mail: zhaoxl@tyust.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61601318);山西省自然科学基金(202103021224278);山西省回国留学人员科研项目(2020-128)
- DOI：10.12263/DZXB.20240253
  中图分类号： TP391.4;
- 收稿：2024-03-20，
  
  修回：2024-06-24，
  
  纸质出版：2024-12-25
- 稿件说明：
移动端阅览
陈佳妮, 武迎春, 吕天琪, 等. 基于穿插特征更新的光场图像超分网络[J]. 电子学报, 2024, 52(12): 4113-4124.

CHEN Jia-ni, WU Ying-chun, LÜ Tian-qi, et al. Cross Feature Updating-Based Network for Light Field Image Super Resolution[J]. Acta Electronica Sinica, 2024, 52(12): 4113-4124.
陈佳妮, 武迎春, 吕天琪, 等. 基于穿插特征更新的光场图像超分网络[J]. 电子学报, 2024, 52(12): 4113-4124. DOI：10.12263/DZXB.20240253

CHEN Jia-ni, WU Ying-chun, LÜ Tian-qi, et al. Cross Feature Updating-Based Network for Light Field Image Super Resolution[J]. Acta Electronica Sinica, 2024, 52(12): 4113-4124. DOI：10.12263/DZXB.20240253

摘要

针对光场图像空间分辨率低的问题，本文搭建了基于穿插特征更新的光场图像超分网络，获得更高质量的光场子孔径图像阵列.网络的浅层特征提取部分采用3分支结构，设计了并行残差块从不同形式的光场数据中提取空间特征和角度特征.深层特征提取采用穿插特征更新结构，设计了特征对齐交互模块、自注意力特征交互模块和空间特征增强模块，实现了空间特征与角度特征逐级融合更新.数据重构部分通过交替使用多尺度残差块和通道注意力块将逐级更新后的空间信息融合，最终经数据上采样得到超分图像.所提网络在充分挖掘、补充空间、角度特征的基础上，采用逐级融合、更新、增强机制，实现不同层次空间信息的收集，获得更好的超分效果.对比实验验证了所提网络的优越性，在5组公开光场数据集上，本文搭建的网络在4倍超分任务下平均峰值信噪比（Peak Signal-to-Noise Ratio， PSNR）值达到32.31 dB，较现有网络表现出更好的超分性能.

Abstract

Addressing the issue of low spatial resolution in light field image

cross feature updating-based network for light field image super resolution is built in this paper to generate a higher quality array of light field sub-aperture images. In this work

a 3-branch structure is adopted for shallow feature extraction. In order to extract spatial and angular features from different forms of light field data

parallel residual block is designed. A cross feature update structure is used for extract deep features

and feature alignment interaction module

self-attention feature interaction module

and spatial feature enhancement module are designed to achieve step-by-step fusion and updating of spatial and angular features. In the data reconstruction part

the updated spatial information is fused by using multi-scale residual block and channel attention block alternately and finally super-resolution images are obtained through data upsampling. On the basis of fully exploring and supplementing spatial and angular features

the proposed network adopts a step-by-step fusion

update

and enhancement mechanism to gather spatial information at different levels

leading to superior super-resolution results. Comparative experiments demonstrate the effectiveness of the proposed method

with the network achieving an average peak signal-to-noise ratio(PSNR) value of 32.31 dB for 4× tasks on 5 public light field datasets

surpassing the performance of existing networks.

关键词

Keywords

references

TAO M W , HADAP S , MALIK J , et al . Depth from combining defocus and correspondence using light-field cameras [C ] // 2013 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2013 : 673 - 680 .

LIU M D , WU R F , LUO Z C , et al . Fast digital refocusing Fourier ptychographic microscopy method based on convolutional neural network [J ] . Optics Express , 2024 , 32 ( 1 ): 339 - 354 .

WANG Y Q , YANG J G , GUO Y L , et al . Selective light field refocusing for camera arrays using bokeh rendering and superresolution [J ] . IEEE Signal Processing Letters , 2019 , 26 ( 1 ): 204 - 208 .

WU G C , MASIA B , JARABO A , et al . Light field image processing: An overview [J ] . IEEE Journal of Selected Topics in Signal Processing , 2017 , 11 ( 7 ): 926 - 954 .

武迎春 , 王玉梅 , 王安红 , 等 . 基于边缘增强引导滤波的光场全聚焦图像融合 [J ] . 电子与信息学报 , 2020 , 42 ( 9 ): 2293 - 2301 .

WU Y C , WANG Y M , WANG A H , et al . Light field all-in-focus image fusion based on edge enhanced guided filtering [J ] . Journal of Electronics & Information Technology , 2020 , 42 ( 9 ): 2293 - 2301 . (in Chinese)

TIAN C R , PAN W H , WANG Z M , et al . DPS-net: Deep polarimetric stereo depth estimation [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 3546 - 3556 .

刘玉轩 , 张力 , 艾海滨 , 等 . 光场相机三维重建研究进展与展望 [J ] . 电子学报 , 2022 , 50 ( 7 ): 1774 - 1792 .

LIU Y X , ZHANG L , AI H B , et al . Progress and prospect of 3D reconstruction based on light field cameras [J ] . Acta Electronica Sinica , 2022 , 50 ( 7 ): 1774 - 1792 . (in Chinese)

周晓清 , 王翔 , 郑锦 , 等 . 基于自适应空间稀疏化的高效多视图立体匹配 [J ] . 电子学报 , 2023 , 51 ( 11 ): 3079 - 3091 .

ZHOU X Q , WANG X , ZHENG J , et al . Adaptive spatial sparsification for efficient multi-view stereo matching [J ] . Acta Electronica Sinica , 2023 , 51 ( 11 ): 3079 - 3091 . (in Chinese)

LAN R S , SUN L , LIU Z B , et al . MADNet: A fast and lightweight network for single-image super resolution [J ] . IEEE Transactions on Cybernetics , 2021 , 51 ( 3 ): 1443 - 1453 .

CHEN H G , HE X H , QING L B , et al . Real-world single image super-resolution: A brief review [J ] . Information Fusion , 2022 , 79 : 124 - 145 .

CHEN Y T , LIU L W , PHONEVILAY V , et al . Image super-resolution reconstruction based on feature map attention mechanism [J ] . Applied Intelligence , 2021 , 51 ( 7 ): 4367 - 4380 .

CHEN Z , ZHANG Y L , GU J J , et al . Dual aggregation transformer for image super-resolution [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 12278 - 12287 .

YOON Y , JEON H G , YOO D , et al . Light-field image super-resolution using convolutional neural network [J ] . IEEE Signal Processing Letters , 2017 , 24 ( 6 ): 848 - 852 .

YEUNG H W F , HOU J H , CHEN X M , et al . Light field spatial super-resolution using deep efficient spatial-angular separable convolution [J ] . IEEE Transactions on Image Processing , 2018 , 28 ( 5 ): 2319 - 2330 .

WANG Y Q , YANG J G , WANG L G , et al . Light field image super-resolution using deformable convolution [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 1057 - 1071 .

WANG S Z , ZHOU T F , LU Y , et al . Detail-preserving transformer for light field image super-resolution [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 3 ): 2522 - 2530 .

LIU G S , YUE H J , WU J M , et al . Intra-inter view interaction network for light field image super-resolution [J ] . IEEE Transactions on Multimedia , 2021 , 25 : 256 - 266 .

WANG Y Q , WANG L G , YANG J G , et al . Spatial-angular interaction for light field image super-resolution [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2020 : 290 - 308 .

WANG Y Q , WANG L G , WU G C , et al . Disentangling light fields for super-resolution and disparity estimation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 1 ): 425 - 443 .

ZHANG S , LIN Y F , SHENG H . Residual networks for light field image super-resolution [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 11038 - 11047 .

ZHANG S , CHANG S , LIN Y F . End-to-end light field spatial super-resolution network using multiple epipolar geometry [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 5956 - 5968 .

HONAUER K , JOHANNSEN O , KONDERMANN D , et al . A dataset and evaluation methodology for depth estimation on 4D light fields [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2017 : 19 - 34 .

WANNER S , MEISTER S , GOLDLÜCKE B . Datasets and benchmarks for densely sampled 4D light fields [C ] // International Symposium on Vision, Modeling, and Visualization . Switzerland : Computer Graphics Forum , 2013 : 1 - 8 .

RERÁBEK M , EBRAHIMI T . New light field image dataset [C ] // 8th International Conference on Quality of Multimedia Experience . Lisbon : Multimedia Signal Processing Laboratory Press 2016 : 1 - 2 .

LE PENDU M , JIANG X R , GUILLEMOT C . Light field inpainting propagation via low rank matrix completion [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 4 ): 1981 - 1993 .

VAISH V , ADAMS A . The (new) stanford light field archive [EB/OL ] . ( 2008-06-07 )[ 2023-12-20 ] . http://lightfield.stanford.edu/ http://lightfield.stanford.edu/ .

LIM B , SON S , KIM H , et al . Enhanced deep residual networks for single image super-resolution [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2017 : 1132 - 1140 .

ZHANG Y L , LI K P , LI K , et al . Image super-resolution using very deep residual channel attention networks [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2018 : 294 - 310 .

LIU G S , YUE H J , WU J M , et al . Efficient light field angular super-resolution with sub-aperture feature learning and macro-pixel upsampling [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 6588 - 6600 .

：作者简介：

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度学习的单帧图像超分辨率重建综述

基于配准的肺4D-CT图像超分辨率重建

基于多点信息统计法的土地覆盖图像超分辨率重建方法