Pansharpening by Jointing Spectral Mapping and Detail Injection

LIU Hong-yi; ZHAO Xiao-fei; HAN Hai-bo; DU Jenny Qian; WU Ze-bin; WEI Zhi-hui

doi:10.12263/DZXB.20210857

您当前的位置：

首页 >

文章列表页 >

Pansharpening by Jointing Spectral Mapping and Detail Injection

PAPERS | 更新时间：2025-12-08

- Pansharpening by Jointing Spectral Mapping and Detail Injection
- ACTA ELECTRONICA SINICA Vol. 51, Issue 6, Pages: 1527-1540(2023)
- 作者机构：
  
  1.南京理工大学数学与统计学院，江苏南京 210094
  2.密西西比州立大学电气与计算机工程系，密西西比州斯塔克维尔 39762
  3.南京理工大学计算机工程与科学学院，江苏南京 210094
- 作者简介：
- 基金信息：
  
  General program of National Natural Science Foundation of China(61971223;61871226);Special Funds for Basic Scientific Research Business Expenses of Central Universities(JSGP202204)
- DOI：10.12263/DZXB.20210857
  CLC： TP391;
- Received：06 July 2021，
  
  Revised：2022-09-05，
  
  Published：25 June 2023
- 稿件说明：
移动端阅览
刘红毅,赵肖飞,韩海波等.基于光谱映射和细节注入的Pansharpening[J].电子学报,2023,51(06):1527-1540.

LIU Hong-yi,ZHAO Xiao-fei,HAN Hai-bo,et al.Pansharpening by Jointing Spectral Mapping and Detail Injection[J].ACTA ELECTRONICA SINICA,2023,51(06):1527-1540.
刘红毅,赵肖飞,韩海波等.基于光谱映射和细节注入的Pansharpening[J].电子学报,2023,51(06):1527-1540. DOI： 10.12263/DZXB.20210857.

LIU Hong-yi,ZHAO Xiao-fei,HAN Hai-bo,et al.Pansharpening by Jointing Spectral Mapping and Detail Injection[J].ACTA ELECTRONICA SINICA,2023,51(06):1527-1540. DOI： 10.12263/DZXB.20210857.

摘要

Pansharpening通过在多光谱图像中注入全色图像的空间细节，从而获得高分辨率的多光谱图像.但细节注入的同时，可能会引起光谱失真.为了提高融合图像的光谱保真，本文提出了一种空谱结合的Pansharpening方法.充分利用全色和多光谱图像之间潜在的光谱关系，对全色图像的光谱信息进行增强，继而将增强后的全色图像细节注入多光谱图像中，并建立了统一的变分Pansharpening模型，同时实现了融合图像的光谱信息保持和空间结构细节增强.在不同数据集上进行的数值实验表明，相比现有的Pansharpening方法，本文所提方法具有比较好的融合效果，尤其在光谱保真方面，更具有一定的优势.

Abstract

Pansharpening aims at improving spatial resolution and retaining spectral resolution for multispectral (MS) image with an aided panchromatic (PAN) image. Most of the existing methods improve the spatial resolution of MS image by injecting spatial details acquired from the PAN image

which may lead to spectral distortion. This paper proposes a joint spectral-spatial pansharpening method. By exploiting the underlying spectral correlation between the PAN and MS images

a spectral-enhanced PAN (SPAN) image is produced. Then the detail of the SPAN image is injected into the MS image

which leads to a unified variational pansharpening model to obtain a fused image with both spatial structure-enhanced and spectral-preserved. Experiments on different datasets confirm the effectiveness of the proposed pansharpening method

especially in the fidelity of spectra.

关键词

Keywords

references

RICHARDS J A , JIA X P . Using suitable neighbors to augment the training set in hyperspectral maximum likelihood classification [J]. IEEE Geoscience and Remote Sensing Letters , 2008 , 5 ( 4 ): 774 - 777 .

SUN L , MA C Y , CHEN Y J , et al . Adjacent superpixel-based multiscale spatial-spectral kernel for hyperspectral classification [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2019 , 12 ( 6 ): 1905 - 1919 .

SUN L , ZHAN T M , WU Z B , et al . Hyperspectral mixed denoising via spectral difference-induced total variation and low-rank approximation [J]. Remote Sensing , 2018 , 10 ( 12 ): 1956 .

HONG D F , YOKOYA N , GE N , et al . Learnable manifold alignment (LeMA): A semi-supervised cross-modality learning framework for land cover and land use classification [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2019 , 147 : 193 - 205 .

CHAVEZ JR P S , SIDES S C , ANDERSON J A . Comparison of three different methods to merge multiresolution and multispectral data: LANDSAT TM and SPOT panchromatic [J]. Photogrammetric Engineering & Remote Sensin , 1991 , 57 ( 3 ): 265 - 303 .

KANATSOULIS C I , FU X , SIDIROPOULOS N D , et al . Hyperspectral super-resolution: A coupled tensor factorization approach [J]. IEEE Transactions on Signal Processing , 2018 , 66 ( 24 ): 6503 - 6517 .

GHASSEMIAN H . A review of remote sensing image fusion methods [J]. Information Fusion , 2016 , 32 : 75 - 89 .

ALPARONE L , WALD L , CHANUSSOT J , et al . Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest [J]. IEEE Transactions on Geoscience and Remote Sensing , 2007 , 45 ( 10 ): 3012 - 3021 .

LONCAN L , DE ALMEIDA L B , BIOUCAS-DIAS J M , et al . Hyperspectral pansharpening: A review [J]. IEEE Geoscience and Remote Sensing Magazine , 2015 , 3 ( 3 ): 27 - 46 .

VIVONE G , ALPARONE L , CHANUSSOT J , et al . A critical comparison of pansharpening algorithms [C]// 2014 IEEE Geoscience and Remote Sensing Symposium . Quebec City : IEEE , 2014 : 191 - 194 .

MENG X C , SHEN H F , LI H F , et al . Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges [J]. Information Fusion , 2019 , 46 : 102 - 113 .

GARZELLI A . A review of image fusion algorithms based on the super-resolution paradigm [J]. Remote Sensing , 2016 , 8 ( 10 ): 797 .

XIONG Z W , SHI Z , LI H Q , et al . HSCNN: CNN-based hyperspectral image recovery from spectrally undersampled projections [C]// 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) . Venice : IEEE , 2018 : 518 - 525 .

HUANG W , XIAO L , WEI Z H , et al . A new pan-sharpening method with deep neural networks [J]. IEEE Geoscience and Remote Sensing Letters , 2015 , 12 ( 5 ): 1037 - 1041 .

MASI G , COZZOLINO D , VERDOLIVA L , et al . Pansharpening by convolutional neural networks [J]. Remote Sensing , 2016 , 8 ( 7 ): 594 - 615 .

WEI Y C , YUAN Q Q , MENG X C , et al . Multi-scale-and-depth convolutional neural network for remote sensed imagery pan-sharpening [C]// 2017 IEEE International Geoscience and Remote Sensing Symposium(IGARSS) . Fort Worth : IEEE , 2017 : 3413 - 3416 .

HE L , RAO Y Z , LI J , et al . Pansharpening via detail injection based convolutional neural networks [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2019 , 12 ( 4 ): 1188 - 1204 .

CHAVEZ P , KWATENG A . Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis [J]. Photogrammetric Engineering and Remote Sensing , 1989 , 55 ( 3 ): 339 - 348 .

LABEN C A , BROWER B V . Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening : 09/069232 [P]. 2000-01-04 .

AIAZZI B , BARONTI S , SELVA M . Improving component substitution pansharpening through multivariate regression of MS+Pan data [J]. IEEE Transactions on Geoscience and Remote Sensing , 2007 , 45 ( 10 ): 3230 - 3239 .

VIVONE G . Robust band-dependent spatial-detail approaches for panchromatic sharpening [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 9 ): 6421 - 6433 .

GARZELLI A , NENCINI F , CAPOBIANCO L . Optimal MMSE pan sharpening of very high resolution multispectral images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2008 , 46 ( 1 ): 228 - 236 .

CARPER W , LILLESAND T , KIEFFER R . The use of intensity-hue-saturation transformations for merging SPOT panchromaticand multispectral image data [J]. Photogrammetric Engineering and Remote Sensing , 1990 , 56 ( 4 ): 459 - 467 .

GHAHREMANI M , GHASSEMIAN H . Nonlinear IHS: A promising method for pan-sharpening [J]. IEEE Geoscience and Remote Sensing Letters , 2016 , 13 ( 11 ): 1606 - 1610 .

LI Q L , YANG X M , WU W , et al . Pansharpening multispectral remote-sensing images with guided filter for monitoring impact of human behavior on environment [J]. Concurrency and Computation: Practice and Experience , 2021 , 33 ( 4 ): e5074 .

JIAN L H , YANG X M , WU W , et al . Pansharpening using a guided image filter based on dual-scale detail extraction [J]. Journal of Ambient Intelligence and Humanized Computing , 2018 , 9 : 1 - 15 .

HE K M , SUN J , TANG X O . Guided image filtering [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 6 ): 1397 - 1409 .

KANG X D , LI S T , BENEDIKTSSON J A . Spectral-spatial hyperspectral image classification with edge-preserving filtering [J]. IEEE Transactions on Geoscience and Remote Sensing , 2014 , 52 ( 5 ): 2666 - 2677 .

LIU J G . Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details [J]. International Journal of Remote Sensing , 2000 , 21 ( 18 ): 3461 - 3472 .

MALLAT S G . A theory for multiresolution signal decomposition: The wavelet representation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 1989 , 11 ( 7 ): 674 - 693 .

STARCK J L , FADILI J , MURTAGH F . The undecimated wavelet decomposition and its reconstruction [J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society , 2007 , 16 ( 2 ): 297 - 309 .

SHENSA M J . The discrete wavelet transform: Wedding the a trous and Mallat algorithms [J]. IEEE Transactions on Signal Processing , 1992 , 40 ( 10 ): 2464 - 2482 .

CHOI M , KIM R Y , NAM M R , et al . Fusion of multispectral and panchromatic Satellite images using the curvelet transform [J]. IEEE Geoscience and Remote Sensing Letters , 2005 , 2 ( 2 ): 136 - 140 .

DO M N , VETTERLI M . The contourlet transform: An efficient directional multiresolution image representation [J]. IEEE Transactions on Image Processing , 2005 , 14 ( 12 ): 2091 - 2106 .

LAPORTERIE F , AMRAM O , FLOUZAT G , et al . Data fusion thanks to an improved morphological pyramid approach: Comparison loop on simulated images and application to SPOT 4 data [C]// IEEE 2000 International Geoscience and Remote Sensing Symposium . Honolulu : IEEE , 2002 : 2117 - 2119 .

AIAZZI B , ALPARONE L , BARONTI S , et al . MTF-tailored multiscale fusion of high-resolution MS and pan imagery [J]. Photogrammetric Engineering & Remote Sensing , 2006 , 72 ( 5 ): 591 - 596 .

VIVONE G , RESTAINO R , CHANUSSOT J . Full scale regression-based injection coefficients for panchromatic sharpening [J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society , 2018 , 27 ( 7 ): 3418 - 3431 .

VIVONE G , MARANO S , CHANUSSOT J . Pansharpening: Context-based generalized Laplacian Pyramids by robust regression [J]. IEEE Transactions on Geoscience and Remote Sensing , 2020 , 58 ( 9 ): 6152 - 6167 .

LIU P F , XIAO L , LI T . A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 3 ): 1788 - 1802 .

BALLESTER C , CASELLES V , IGUAL L , et al . A variational model for P+XS image fusion [J]. International Journal of Computer Vision , 2006 , 69 ( 1 ): 43 - 58 .

JIANG C , ZHANG H Y , SHEN H F , et al . A practical compressed sensing-based pan-sharpening method [J]. IEEE Geoscience and Remote Sensing Letters , 2012 , 9 ( 4 ): 629 - 633 .

GUO M , ZHANG H Y , LI J Y , et al . An online coupled dictionary learning approach for remote sensing image fusion [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2014 , 7 ( 4 ): 1284 - 1294 .

FU X Y , LIN Z H , HUANG Y , et al . A variational pan-sharpening with local gradient constraints [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach : IEEE , 2020 : 10257 - 10266 .

ZHU X X , BAMLER R . A sparse image fusion algorithm with application to pan-sharpening [J]. IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 5 ): 2827 - 2836 .

AHARON M , ELAD M , BRUCKSTEIN A . K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation [J]. IEEE Transactions on Signal Processing , 2006 , 54 ( 11 ): 4311 - 4322 .

HUANG W , XIAO L , LIU H Y , et al . Hyperspectral imagery super-resolution by compressive sensing inspired dictionary learning and spatial-spectral regularization [J]. Sensors , 2015 , 15 ( 1 ): 2041 - 2058 .

LI S T , YIN H T , FANG L Y . Remote sensing image fusion via sparse representations over learned dictionaries [J]. IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 9 ): 4779 - 42016 .

HE W , ZHANG H Y , ZHANG L P , et al . Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration [J]. IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 1 ): 178 - 188 .

XU Y , WU Z B , CHANUSSOT J , et al . Nonlocal patch tensor sparse representation for hyperspectral image super-resolution [J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society , 2019 , 28 ( 6 ): 3034 - 3047 .

DURAN J , BUADES A , COLL B , et al . Pansharpening by a nonlocal channel-decoupled variational method [C]// 2016 IEEE International Conference on Image Processing(ICIP) . Phoenix : IEEE , 2016 : 4339 - 4343 .

ZHANG H Y , HE W , ZHANG L P , et al . Hyperspectral image restoration using low-rank matrix recovery [J]. IEEE Transactions on Geoscience and Remote Sensing , 2014 , 52 ( 8 ): 4729 - 4743 .

LIU H Y , SUN P P , DU Q , et al . Hyperspectral image restoration based on low-rank recovery with a local neighborhood weighted spectral-spatial total variation model [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 3 ): 1409 - 1422 .

ZHOU J , KWAN C , BUDAVARI B . Hyperspectral image super-resolution: A hybrid color mapping approach [J]. Journal of Applied Remote Sensing , 2016 , 10 ( 3 ): 035024 .

VITALE S , SCARPA G . A detail-preserving cross-scale learning strategy for CNN-based pansharpening [J]. Remote Sensing , 2020 , 12 ( 3 ): 348 .

WANG D , LI Y , MA L , et al . Going deeper with densely connected convolutional neural networks for multispectral pansharpening [J]. Remote Sensing , 2019 , 11 ( 22 ): 2608 .

ZHAO X F , LIU H Y , ZHANG J , et al . A spectral mapping based intensity modulation for pan-sharpening [C]// IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium . Yokohama : IEEE , 2019 : 843 - 846 .

LI D , SHEN J , LU Y , et al . A class of alternating linearization algorithms for nonsmooth convex optimization [J]. Acta Mathematicae Applicatae Sinica , English Series, 2019 , 35 ( 2 ): 435 - 443 .

GAO X , CAI X J , HAN D R . A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems [J]. Journal of Global Optimization , 2020 , 76 ( 4 ): 863 - 887 .

BOLTE J , SABACH S , TEBOULLE M . Proximal alternating linearized minimization for nonconvex and nonsmooth problems [J]. Mathematical Programming , 2014 , 146 ( 1 ): 459 - 494 .

WALD L , RANCHIN T , MANGOLINI M . Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images [J]. Photogrammetric Engineering and Remote Sensing , 1997 , 63 ( 6 ): 691 - 699 .

OTAZU X , GONZALEZ-AUDICANA M , FORS O , et al . Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods [J]. IEEE Transactions on Geoscience and Remote Sensing , 2005 , 43 ( 10 ): 2376 - 2385 .

YUHAS R H , GOETZ ALEXANDER F H , BOARDMAN J W . Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm [C]// Summaries of the Third Annual JPL Airborne Geoscience Workshop . California : Jet Propulsion Laboratory , 1992 : 147 - 149 .

WALD L . Data fusion: definitions and architectures - fusion of images of different spatial resolutions [EB/OL]. ( 2002 )[2021]. https://hal-mines-paristech.archives-ouvertes.fr/hal-00464703 https://hal-mines-paristech.archives-ouvertes.fr/hal-00464703 .

WANG Z , BOVIK A C . A universal image quality index [J]. IEEE Signal Processing Letters , 2002 , 9 ( 3 ): 81 - 84 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-Spectral Image Fusion Method Based on Nonsubsampled Three Channels Nonseparable Symmetrical Wavelets

Multi-Spectral Image Fusion Method Based on Sampled Two Channel Non-Separable Wavelets

Multiscale Remote Sensing Image Fusion Algorithm Based on Variational Segmentation

Related Author

PENG Jia-xiong

LIU Bin

LIU Wei-jie

HU Fu-qiang

ZHU Qing

ZHANG Yong-mei

JI Yan

MA Li

Related Institution

Institute of Image Recognition and Artificial IntelligenceHuazhong University of Science and TechnologyWuhanHubei 430074China

School of Mathematics and Computer ScienceHubei UniversityWuhanHubei 430062China

Institute of Image Recognition and Artificial Intelligence,Huazhong University of Science and Technology

School of Mathematics and Computer Science, Hubei University

Computer School, Wuhan University

⁰