UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation

JIA Xiu-yi; LIN Qiao-wan-ni; ZHENG Zhuo-ran; SHI Zheng-hao

doi:10.12263/DZXB.20230840

您当前的位置：

首页 >

文章列表页 >

UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation

PAPERS | 更新时间：2025-12-08

- UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation
- ACTA ELECTRONICA SINICA Vol. 53, Issue 1, Pages: 238-247(2025)
- 作者机构：
  
  1.南京理工大学计算机科学与工程学院，江苏南京 210094
  2.西安理工大学计算机科学与工程学院，陕西西安 710048
- 作者简介：
- 基金信息：
  
  National Natural Science of China(62176123)
- DOI：10.12263/DZXB.20230840
  CLC： TP397.4;
- Received：05 September 2023，
  
  Revised：2024-07-06，
  
  Published：25 January 2025
- 稿件说明：
移动端阅览
贾修一, 林乔万尼, 郑卓然, 等. 基于稠密局部-全局特征融合的超高清多曝光图像融合方法[J]. 电子学报, 2025, 53(01): 238-247.

JIA Xiu-yi, LIN Qiao-wan-ni, ZHENG Zhuo-ran, et al. UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation[J]. Acta Electronica Sinica, 2025, 53(01): 238-247.
贾修一, 林乔万尼, 郑卓然, 等. 基于稠密局部-全局特征融合的超高清多曝光图像融合方法[J]. 电子学报, 2025, 53(01): 238-247. DOI：10.12263/DZXB.20230840

JIA Xiu-yi, LIN Qiao-wan-ni, ZHENG Zhuo-ran, et al. UHD Multi-Exposure Image Fusion via Dense Local-Global Feature Aggregation[J]. Acta Electronica Sinica, 2025, 53(01): 238-247. DOI：10.12263/DZXB.20230840

摘要

随着超高清（Ultra-High-Definition，UHD）成像技术的应用，生成高质量的UHD图像通常需要融合多幅曝光水平不同的UHD图像.然而，目前基于深度学习的多曝光图像融合方法直接融合从不同曝光水平的图像中提取的特征图，未能充分利用不同曝光级别图像中的特征信息，而这些特征信息对于获得良好的多曝光融合结果至关重要.为解决这一问题，我们提出了一种新颖的UHD多曝光图像融合方法，该方法结合了图像的局部和长距离依赖特征，旨在挖掘不同曝光级别图像之间的依赖关系，提取出更高阶的语义和特征.进而，利用不同级别的短连接来聚合不同粒度的特征.最后，为了过滤带噪声的特征，我们还提出了带有门控机制的多层感知器来生成高质量的超高清图像.为了更好地展示实验结果，我们还针对多曝光融合任务建立了一个UHD图像数据集.实验结果表明，在单个显存24G的GPU上执行UHD多曝光图像融合任务时，我们的方法明显优于现有方法.

Abstract

With the deployment of ultra-high-definition (UHD) imaging technology

generating high-quality UHD images typically involves fusing multiple UHD images with varying exposure levels. However

current multi-exposure image fusion (MEF) methods based on deep learning perform direct fusion of feature maps extracted from images with different exposure levels. These methods fail to fully exploit the feature information in images with varying exposure levels

which is essential for achieving successful MEF outcomes. To address this problem

we develop a UHD multi-exposure image fusion approach that incorporates both local and long-range characteristics of images

and it aims to mine the dependencies of images with different exposure levels. By enforcing translation invariance and self-attention on images with varying exposure levels

we can extract higher-level semantics and features. Furthermore

we aggregate the resulting features of different granularity by utilizing shortcut connections at various levels. Finally

we propose the Gate-MLP with a gating mechanism for filtering features with noise to generate a high-quality UHD image. To better demonstrate the work for UHD MEF task

we also establish a UHD image dataset for MEF task. Extensive experimental results demonstrate that ourapproach significantly outperforms existing approaches for UHD multi-exposure image fusion task on a single 24G RAM GPU.

关键词

Keywords

references

BHANDARI A K , SUBRAMANI B , VELUCHAMY M . Multi-exposure optimized contrast and brightness balance color image enhancement [J ] . Digital Signal Processing , 2022 , 123 : 103406 .

DENG X , ZHANG Y T , XU M , et al . Deep coupled feedback network for joint exposure fusion and image super-resolution [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 3098 - 3112 .

张雨童 , 邓欣 , 徐迈 . 动态场景下深度自监督多曝光图像融合方法 [J ] . 电子学报 , 2024 , 52 ( 1 ): 264 - 273 .

ZHANG Y T , DENG X , XU M . Deep self-supervised multi-exposure image fusion for dynamic scenes [J ] . Acta Electronica Sinica , 2024 , 52 ( 1 ): 264 - 273 . (in Chinese)

QU L H , LIU S L , WANG M N , et al . TransMEF: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 2 ): 2126 - 2134 .

VASWANI A , SHAZEER N , PAEMAR N , et al . Attention is all you need [C ] // Proceedings of Advances in Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

QI G Q , CHANG L , LUO Y Q , et al . A precise multi-exposure image fusion method based on low-level features [J ] . Sensors , 2020 , 20 ( 6 ): 1597 .

WANG Q T , CHEN W H , WU X M , et al . Detail-enhanced multi-scale exposure fusion in YUV color space [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 30 ( 8 ): 2418 - 2429 .

LIU J Y , SHANG J J , LIU R S , et al . Halder: Hierarchical attention-guided learning with detail-refinement for multi-exposure image fusion [C ] // 2021 IEEE International Conference on Multimedia and Expo (ICME) . Piscataway : IEEE , 2021 : 1 - 6 .

DENG X , DRAGOTTI P L . Deep convolutional neural network for multi-modal image restoration and fusion [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 10 ): 3333 - 3348 .

YANG Z G , CHEN Y P , LE Z L , et al . GANFuse: A novel multi-exposure image fusion method based on generative adversarial networks [J ] . Neural Computing and Applications , 2021 , 33 ( 11 ): 6133 - 6145 .

JUNG H , KIM Y , JANG H , et al . Unsupervised deep image fusion with structure tensor representations [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 3845 - 3858 .

HU X B , REN W Q , YU K C , et al . Pyramid architecture search for real-time image deblurring [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 4278 - 4287 .

LIANG J , ZENG H , ZHANG L . High-resolution photorealistic image translation in real-time: A Laplacian pyramid translation network [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 9392 - 9400 .

LIN S C , RYABTSEV A , SENGUPTA S , et al . Real-time high-resolution background matting [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 8758 - 8767 .

LIN Q W N , ZHENG Z R , JIA X Y . UHD low-light image enhancement via interpretable bilateral learning [J ] . Information Sciences , 2022 , 608 : 1401 - 1415 .

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 . Cham : Springer International Publishing , 2015 : 234 - 241 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer: Efficient transformer for high-resolution image restoration [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 5718 - 5729 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16x16 words: Transformers for image recognition at scale [C ] // Proceedings of the International Conference on Learning Representations , Dublin : ICLR , 2021 : 1 - 21 .

LIU Y , WANG Z F . Dense SIFT for ghost-free multi-exposure fusion [J ] . Journal of Visual Communication and Image Representation , 2015 , 31 : 208 - 224 .

LI H , MA K D , YONG H W , et al . Fast multi-scale structural patch decomposition for multi-exposure image fusion [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 5805 - 5816 .

MA K D , WANG Z . Multi-exposure image fusion: A patch-wise approach [C ] // 2015 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2015 : 1717 - 1721 .

LEE S H , PARK J S , CHO N I . A multi-exposure image fusion based on the adaptive weights reflecting the relative pixel intensity and global gradient [C ] // 2018 25th IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2018 : 1737 - 1741 .

ZHANG Y , LIU Y , SUN P , et al . IFCNN: A general image fusion framework based on convolutional neural network [J ] . Information Fusion , 2020 , 54 : 99 - 118 .

MA K D , DUANMU Z F , ZHU H W , et al . Deep guided learning for fast multi-exposure image fusion [J ] . IEEE Transactions on Image Processing , 2019 , 29 : 2808 - 2819 .

XU H , MA J Y , JIANG J J , et al . U2Fusion: A unified unsupervised image fusion network [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 1 ): 502 - 518 .

CAI J R , GU S H , ZHANG L , et al . Learning a deep single image contrast enhancer from multi-exposure images [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 4 ): 2049 - 2062 .

LV F F , LI Y , LU F . Attention guided low-light image enhancement with a large scale low-light simulation dataset [J ] . International Journal of Computer Vision , 2021 , 129 ( 7 ): 2175 - 2193 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Shared Super-Resolution Dual-Branch Network for Spatiotemporal Fusion of Remote Sensing Images

Deep Self-Supervised Multi-Exposure Image Fusion for Dynamic Scenes

A Real-Time Urban Area Detection Algorithm Based on Feature Location Optimization and Integration

Adaptive Group Masked Graph Convolution Network for Pedestrian Trajectory Prediction

Hierarchical Identity-Based Signature Scheme Based on SM9

Related Author

FANG Shuai

ZHANG Xiao-xi

ZHANG Jing

ZHANG Yu-tong

DENG Xin

XU Mai

LIU Yan-hong

WEI Xue-ye

Related Institution

School of Computer and Information, Hefei University of Technology

Anhui Province Key Laboratory of Industry Safety and Emergency Technology

School of Electronic and Information Engineering， Beihang University

School of Cyber Science and Technology， Beihang University

China Telecom Xinjiang Telecom Co.Ltd.UrumqiXinjiang 830026China

⁰