基于稠密局部-全局特征融合的超高清多曝光图像融合方法

贾修一; 林乔万尼; 郑卓然; 石争浩

doi:10.12263/DZXB.20230840

您当前的位置：

首页 >

文章列表页 >

基于稠密局部-全局特征融合的超高清多曝光图像融合方法

学术论文 | 更新时间：2025-12-08

- 基于稠密局部-全局特征融合的超高清多曝光图像融合方法
- UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation
- 电子学报 2025年53卷第1期页码：238-247
- 作者机构：
  
  1.南京理工大学计算机科学与工程学院，江苏南京 210094
  2.西安理工大学计算机科学与工程学院，陕西西安 710048
- 作者简介：
  
  [ "贾修一男，1983年出生于山东省日照市.教授，博士生导师.主要研究方向为机器学习、数据挖掘、计算机视觉. E-mail: jiaxy@njust.edu.cn" ]
  [ "林乔万尼男，1999年出生于浙江省温州市.毕业于南京理工大学并获硕士学位.主要研究方向为图像处理、计算机视觉.E-mail: qwnlin@njust.edu.cn" ]
  [ "郑卓然男，1992年出生于山东省枣庄市.毕业于南京理工大学并获博士学位.主要研究方向为图像增强、计算机视觉.E-mail: zhengzr@njust.edu.cn" ]
  [ "石争浩男，1968年出生于陕西省富平县.教授，博士生导师.主要研究方向为机器视觉、医学图像处理及机器学习.E-mail: ylshi@xaut.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62176123)
- DOI：10.12263/DZXB.20230840
  中图分类号： TP397.4;
- 收稿：2023-09-05，
  
  修回：2024-07-06，
  
  纸质出版：2025-01-25
- 稿件说明：
移动端阅览
贾修一, 林乔万尼, 郑卓然, 等. 基于稠密局部-全局特征融合的超高清多曝光图像融合方法[J]. 电子学报, 2025, 53(01): 238-247.

JIA Xiu-yi, LIN Qiao-wan-ni, ZHENG Zhuo-ran, et al. UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation[J]. Acta Electronica Sinica, 2025, 53(01): 238-247.
贾修一, 林乔万尼, 郑卓然, 等. 基于稠密局部-全局特征融合的超高清多曝光图像融合方法[J]. 电子学报, 2025, 53(01): 238-247. DOI：10.12263/DZXB.20230840

JIA Xiu-yi, LIN Qiao-wan-ni, ZHENG Zhuo-ran, et al. UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation[J]. Acta Electronica Sinica, 2025, 53(01): 238-247. DOI：10.12263/DZXB.20230840

摘要

随着超高清（Ultra-High-Definition，UHD）成像技术的应用，生成高质量的UHD图像通常需要融合多幅曝光水平不同的UHD图像.然而，目前基于深度学习的多曝光图像融合方法直接融合从不同曝光水平的图像中提取的特征图，未能充分利用不同曝光级别图像中的特征信息，而这些特征信息对于获得良好的多曝光融合结果至关重要.为解决这一问题，我们提出了一种新颖的UHD多曝光图像融合方法，该方法结合了图像的局部和长距离依赖特征，旨在挖掘不同曝光级别图像之间的依赖关系，提取出更高阶的语义和特征.进而，利用不同级别的短连接来聚合不同粒度的特征.最后，为了过滤带噪声的特征，我们还提出了带有门控机制的多层感知器来生成高质量的超高清图像.为了更好地展示实验结果，我们还针对多曝光融合任务建立了一个UHD图像数据集.实验结果表明，在单个显存24G的GPU上执行UHD多曝光图像融合任务时，我们的方法明显优于现有方法.

Abstract

With the deployment of ultra-high-definition (UHD) imaging technology

generating high-quality UHD images typically involves fusing multiple UHD images with varying exposure levels. However

current multi-exposure image fusion (MEF) methods based on deep learning perform direct fusion of feature maps extracted from images with different exposure levels. These methods fail to fully exploit the feature information in images with varying exposure levels

which is essential for achieving successful MEF outcomes. To address this problem

we develop a UHD multi-exposure image fusion approach that incorporates both local and long-range characteristics of images

and it aims to mine the dependencies of images with different exposure levels. By enforcing translation invariance and self-attention on images with varying exposure levels

we can extract higher-level semantics and features. Furthermore

we aggregate the resulting features of different granularity by utilizing shortcut connections at various levels. Finally

we propose the Gate-MLP with a gating mechanism for filtering features with noise to generate a high-quality UHD image. To better demonstrate the work for UHD MEF task

we also establish a UHD image dataset for MEF task. Extensive experimental results demonstrate that ourapproach significantly outperforms existing approaches for UHD multi-exposure image fusion task on a single 24G RAM GPU.

关键词

Keywords

references

BHANDARI A K , SUBRAMANI B , VELUCHAMY M . Multi-exposure optimized contrast and brightness balance color image enhancement [J ] . Digital Signal Processing , 2022 , 123 : 103406 .

DENG X , ZHANG Y T , XU M , et al . Deep coupled feedback network for joint exposure fusion and image super-resolution [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 3098 - 3112 .

张雨童 , 邓欣 , 徐迈 . 动态场景下深度自监督多曝光图像融合方法 [J ] . 电子学报 , 2024 , 52 ( 1 ): 264 - 273 .

ZHANG Y T , DENG X , XU M . Deep self-supervised multi-exposure image fusion for dynamic scenes [J ] . Acta Electronica Sinica , 2024 , 52 ( 1 ): 264 - 273 . (in Chinese)

QU L H , LIU S L , WANG M N , et al . TransMEF: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 2 ): 2126 - 2134 .

VASWANI A , SHAZEER N , PAEMAR N , et al . Attention is all you need [C ] // Proceedings of Advances in Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

QI G Q , CHANG L , LUO Y Q , et al . A precise multi-exposure image fusion method based on low-level features [J ] . Sensors , 2020 , 20 ( 6 ): 1597 .

WANG Q T , CHEN W H , WU X M , et al . Detail-enhanced multi-scale exposure fusion in YUV color space [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 30 ( 8 ): 2418 - 2429 .

LIU J Y , SHANG J J , LIU R S , et al . Halder: Hierarchical attention-guided learning with detail-refinement for multi-exposure image fusion [C ] // 2021 IEEE International Conference on Multimedia and Expo (ICME) . Piscataway : IEEE , 2021 : 1 - 6 .

DENG X , DRAGOTTI P L . Deep convolutional neural network for multi-modal image restoration and fusion [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 10 ): 3333 - 3348 .

YANG Z G , CHEN Y P , LE Z L , et al . GANFuse: A novel multi-exposure image fusion method based on generative adversarial networks [J ] . Neural Computing and Applications , 2021 , 33 ( 11 ): 6133 - 6145 .

JUNG H , KIM Y , JANG H , et al . Unsupervised deep image fusion with structure tensor representations [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 3845 - 3858 .

HU X B , REN W Q , YU K C , et al . Pyramid architecture search for real-time image deblurring [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4278 - 4287 .

LIANG J , ZENG H , ZHANG L . High-resolution photorealistic image translation in real-time: A Laplacian pyramid translation network [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 9392 - 9400 .

LIN S C , RYABTSEV A , SENGUPTA S , et al . Real-time high-resolution background matting [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 8758 - 8767 .

LIN Q W N , ZHENG Z R , JIA X Y . UHD low-light image enhancement via interpretable bilateral learning [J ] . Information Sciences , 2022 , 608 : 1401 - 1415 .

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 . Cham : Springer International Publishing , 2015 : 234 - 241 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer: Efficient transformer for high-resolution image restoration [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 5718 - 5729 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16x16 words: Transformers for image recognition at scale [C ] // Proceedings of the International Conference on Learning Representations , Dublin : ICLR , 2021 : 1 - 21 .

LIU Y , WANG Z F . Dense SIFT for ghost-free multi-exposure fusion [J ] . Journal of Visual Communication and Image Representation , 2015 , 31 : 208 - 224 .

LI H , MA K D , YONG H W , et al . Fast multi-scale structural patch decomposition for multi-exposure image fusion [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 5805 - 5816 .

MA K D , WANG Z . Multi-exposure image fusion: A patch-wise approach [C ] // 2015 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2015 : 1717 - 1721 .

LEE S H , PARK J S , CHO N I . A multi-exposure image fusion based on the adaptive weights reflecting the relative pixel intensity and global gradient [C ] // 2018 25th IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2018 : 1737 - 1741 .

ZHANG Y , LIU Y , SUN P , et al . IFCNN: A general image fusion framework based on convolutional neural network [J ] . Information Fusion , 2020 , 54 : 99 - 118 .

MA K D , DUANMU Z F , ZHU H W , et al . Deep guided learning for fast multi-exposure image fusion [J ] . IEEE Transactions on Image Processing , 2019 , 29 : 2808 - 2819 .

XU H , MA J Y , JIANG J J , et al . U2Fusion: A unified unsupervised image fusion network [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 1 ): 502 - 518 .

CAI J R , GU S H , ZHANG L , et al . Learning a deep single image contrast enhancer from multi-exposure images [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 4 ): 2049 - 2062 .

LV F F , LI Y , LU F . Attention guided low-light image enhancement with a large scale low-light simulation dataset [J ] . International Journal of Computer Vision , 2021 , 129 ( 7 ): 2175 - 2193 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

共享超分的双分支遥感图像时空融合网络

动态场景下深度自监督多曝光图像融合方法

基于特征位置优选整合的快速城区检测算法

基于加权优先级与数据包到达时间的MP-QUIC调度算法

基于多域表征对齐融合的雷达目标无监督异常检测方法