Cross Feature Updating-Based Network for Light Field Image Super Resolution

CHEN Jia-ni; WU Ying-chun; LÜ Tian-qi; LIU Li; ZHAO Xian-ling

doi:10.12263/DZXB.20240253

您当前的位置：

首页 >

文章列表页 >

Cross Feature Updating-Based Network for Light Field Image Super Resolution

PAPERS | 更新时间：2025-12-11

- Cross Feature Updating-Based Network for Light Field Image Super Resolution
- ACTA ELECTRONICA SINICA Vol. 52, Issue 12, Pages: 4113-4124(2024)
- 作者机构：
  
  太原科技大学电子信息工程学院，山西太原 030024
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(61601318);Fundamental Research Program of Shanxi Province(202103021224278);Research Project Supported by Shanxi Scholarship Council of China(2020-128)
- DOI：10.12263/DZXB.20240253
  CLC： TP391.4;
- Received：20 March 2024，
  
  Revised：2024-06-24，
  
  Published：25 December 2024
- 稿件说明：
移动端阅览
陈佳妮, 武迎春, 吕天琪, 等. 基于穿插特征更新的光场图像超分网络[J]. 电子学报, 2024, 52(12): 4113-4124.

CHEN Jia-ni, WU Ying-chun, LÜ Tian-qi, et al. Cross Feature Updating-Based Network for Light Field Image Super Resolution[J]. Acta Electronica Sinica, 2024, 52(12): 4113-4124.
陈佳妮, 武迎春, 吕天琪, 等. 基于穿插特征更新的光场图像超分网络[J]. 电子学报, 2024, 52(12): 4113-4124. DOI：10.12263/DZXB.20240253

CHEN Jia-ni, WU Ying-chun, LÜ Tian-qi, et al. Cross Feature Updating-Based Network for Light Field Image Super Resolution[J]. Acta Electronica Sinica, 2024, 52(12): 4113-4124. DOI：10.12263/DZXB.20240253

摘要

针对光场图像空间分辨率低的问题，本文搭建了基于穿插特征更新的光场图像超分网络，获得更高质量的光场子孔径图像阵列.网络的浅层特征提取部分采用3分支结构，设计了并行残差块从不同形式的光场数据中提取空间特征和角度特征.深层特征提取采用穿插特征更新结构，设计了特征对齐交互模块、自注意力特征交互模块和空间特征增强模块，实现了空间特征与角度特征逐级融合更新.数据重构部分通过交替使用多尺度残差块和通道注意力块将逐级更新后的空间信息融合，最终经数据上采样得到超分图像.所提网络在充分挖掘、补充空间、角度特征的基础上，采用逐级融合、更新、增强机制，实现不同层次空间信息的收集，获得更好的超分效果.对比实验验证了所提网络的优越性，在5组公开光场数据集上，本文搭建的网络在4倍超分任务下平均峰值信噪比（Peak Signal-to-Noise Ratio， PSNR）值达到32.31 dB，较现有网络表现出更好的超分性能.

Abstract

Addressing the issue of low spatial resolution in light field image

cross feature updating-based network for light field image super resolution is built in this paper to generate a higher quality array of light field sub-aperture images. In this work

a 3-branch structure is adopted for shallow feature extraction. In order to extract spatial and angular features from different forms of light field data

parallel residual block is designed. A cross feature update structure is used for extract deep features

and feature alignment interaction module

self-attention feature interaction module

and spatial feature enhancement module are designed to achieve step-by-step fusion and updating of spatial and angular features. In the data reconstruction part

the updated spatial information is fused by using multi-scale residual block and channel attention block alternately and finally super-resolution images are obtained through data upsampling. On the basis of fully exploring and supplementing spatial and angular features

the proposed network adopts a step-by-step fusion

update

and enhancement mechanism to gather spatial information at different levels

leading to superior super-resolution results. Comparative experiments demonstrate the effectiveness of the proposed method

with the network achieving an average peak signal-to-noise ratio(PSNR) value of 32.31 dB for 4× tasks on 5 public light field datasets

surpassing the performance of existing networks.

关键词

Keywords

references

TAO M W , HADAP S , MALIK J , et al . Depth from combining defocus and correspondence using light-field cameras [C ] // 2013 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2013 : 673 - 680 .

LIU M D , WU R F , LUO Z C , et al . Fast digital refocusing Fourier ptychographic microscopy method based on convolutional neural network [J ] . Optics Express , 2024 , 32 ( 1 ): 339 - 354 .

WANG Y Q , YANG J G , GUO Y L , et al . Selective light field refocusing for camera arrays using bokeh rendering and superresolution [J ] . IEEE Signal Processing Letters , 2019 , 26 ( 1 ): 204 - 208 .

WU G C , MASIA B , JARABO A , et al . Light field image processing: An overview [J ] . IEEE Journal of Selected Topics in Signal Processing , 2017 , 11 ( 7 ): 926 - 954 .

武迎春 , 王玉梅 , 王安红 , 等 . 基于边缘增强引导滤波的光场全聚焦图像融合 [J ] . 电子与信息学报 , 2020 , 42 ( 9 ): 2293 - 2301 .

WU Y C , WANG Y M , WANG A H , et al . Light field all-in-focus image fusion based on edge enhanced guided filtering [J ] . Journal of Electronics & Information Technology , 2020 , 42 ( 9 ): 2293 - 2301 . (in Chinese)

TIAN C R , PAN W H , WANG Z M , et al . DPS-net: Deep polarimetric stereo depth estimation [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 3546 - 3556 .

刘玉轩 , 张力 , 艾海滨 , 等 . 光场相机三维重建研究进展与展望 [J ] . 电子学报 , 2022 , 50 ( 7 ): 1774 - 1792 .

LIU Y X , ZHANG L , AI H B , et al . Progress and prospect of 3D reconstruction based on light field cameras [J ] . Acta Electronica Sinica , 2022 , 50 ( 7 ): 1774 - 1792 . (in Chinese)

周晓清 , 王翔 , 郑锦 , 等 . 基于自适应空间稀疏化的高效多视图立体匹配 [J ] . 电子学报 , 2023 , 51 ( 11 ): 3079 - 3091 .

ZHOU X Q , WANG X , ZHENG J , et al . Adaptive spatial sparsification for efficient multi-view stereo matching [J ] . Acta Electronica Sinica , 2023 , 51 ( 11 ): 3079 - 3091 . (in Chinese)

LAN R S , SUN L , LIU Z B , et al . MADNet: A fast and lightweight network for single-image super resolution [J ] . IEEE Transactions on Cybernetics , 2021 , 51 ( 3 ): 1443 - 1453 .

CHEN H G , HE X H , QING L B , et al . Real-world single image super-resolution: A brief review [J ] . Information Fusion , 2022 , 79 : 124 - 145 .

CHEN Y T , LIU L W , PHONEVILAY V , et al . Image super-resolution reconstruction based on feature map attention mechanism [J ] . Applied Intelligence , 2021 , 51 ( 7 ): 4367 - 4380 .

CHEN Z , ZHANG Y L , GU J J , et al . Dual aggregation transformer for image super-resolution [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 12278 - 12287 .

YOON Y , JEON H G , YOO D , et al . Light-field image super-resolution using convolutional neural network [J ] . IEEE Signal Processing Letters , 2017 , 24 ( 6 ): 848 - 852 .

YEUNG H W F , HOU J H , CHEN X M , et al . Light field spatial super-resolution using deep efficient spatial-angular separable convolution [J ] . IEEE Transactions on Image Processing , 2018 , 28 ( 5 ): 2319 - 2330 .

WANG Y Q , YANG J G , WANG L G , et al . Light field image super-resolution using deformable convolution [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 1057 - 1071 .

WANG S Z , ZHOU T F , LU Y , et al . Detail-preserving transformer for light field image super-resolution [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 3 ): 2522 - 2530 .

LIU G S , YUE H J , WU J M , et al . Intra-inter view interaction network for light field image super-resolution [J ] . IEEE Transactions on Multimedia , 2021 , 25 : 256 - 266 .

WANG Y Q , WANG L G , YANG J G , et al . Spatial-angular interaction for light field image super-resolution [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2020 : 290 - 308 .

WANG Y Q , WANG L G , WU G C , et al . Disentangling light fields for super-resolution and disparity estimation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 1 ): 425 - 443 .

ZHANG S , LIN Y F , SHENG H . Residual networks for light field image super-resolution [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 11038 - 11047 .

ZHANG S , CHANG S , LIN Y F . End-to-end light field spatial super-resolution network using multiple epipolar geometry [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 5956 - 5968 .

HONAUER K , JOHANNSEN O , KONDERMANN D , et al . A dataset and evaluation methodology for depth estimation on 4D light fields [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2017 : 19 - 34 .

WANNER S , MEISTER S , GOLDLÜCKE B . Datasets and benchmarks for densely sampled 4D light fields [C ] // International Symposium on Vision, Modeling, and Visualization . Switzerland : Computer Graphics Forum , 2013 : 1 - 8 .

RERÁBEK M , EBRAHIMI T . New light field image dataset [C ] // 8th International Conference on Quality of Multimedia Experience . Lisbon : Multimedia Signal Processing Laboratory Press 2016 : 1 - 2 .

LE PENDU M , JIANG X R , GUILLEMOT C . Light field inpainting propagation via low rank matrix completion [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 4 ): 1981 - 1993 .

VAISH V , ADAMS A . The (new) stanford light field archive [EB/OL ] . ( 2008-06-07 )[ 2023-12-20 ] . http://lightfield.stanford.edu/ http://lightfield.stanford.edu/ .

LIM B , SON S , KIM H , et al . Enhanced deep residual networks for single image super-resolution [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2017 : 1132 - 1140 .

ZHANG Y L , LI K P , LI K , et al . Image super-resolution using very deep residual channel attention networks [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2018 : 294 - 310 .

LIU G S , YUE H J , WU J M , et al . Efficient light field angular super-resolution with sub-aperture feature learning and macro-pixel upsampling [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 6588 - 6600 .

：作者简介：

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Review of Single Image Super-Resolution Reconstruction Based on Deep Learning

Registration Based Super-Resolution Reconstruction for Lung 4D-CT Image

A Super-Resolution Reconstruction Method for Land Cover Maps Using Multiple-Point Statistics

Related Author

WU Jing

YE Xiao-jing

HUANG Feng

CHEN Li-qiong

WANG Zhi-feng

LIU Wen-xi

WU Le-nan

SUN Jin-tao

Related Institution

School of Mechanical Engineering and Automation, Fuzhou University

Advanced Technology Innovation Institute, Fuzhou University

College of Computer and Data Science, Fuzhou University

Department of Radio EngineeringSoutheast UniversityNanjingJiangsu 210096China

School of Electronic Engineering and Optoelectronic TechniquesNanjing University of Science and TechnologyNanjing Jiangsu 210094China

⁰