基于光谱映射和细节注入的Pansharpening

刘红毅; 赵肖飞; 韩海波; Jenny Qian Du; 吴泽彬; 韦志辉

doi:10.12263/DZXB.20210857

您当前的位置：

首页 >

文章列表页 >

基于光谱映射和细节注入的Pansharpening

学术论文 | 更新时间：2025-12-08

- 基于光谱映射和细节注入的Pansharpening
- Pansharpening by Jointing Spectral Mapping and Detail Injection
- 电子学报 2023年51卷第6期页码：1527-1540
- 作者机构：
  
  1.南京理工大学数学与统计学院，江苏南京 210094
  2.密西西比州立大学电气与计算机工程系，密西西比州斯塔克维尔 39762
  3.南京理工大学计算机工程与科学学院，江苏南京 210094
- 作者简介：
  
  [ "刘红毅女，1974年出生于陕西省西安市.南京理工大学数学与统计学院副教授.研究方向为基于图像处理与分析、计算机视觉.E-mail: hyliu@njust.edu.cn" ]
  [ "赵肖飞女，1995年出生于河北省沧州市.南京理工大学数学与统计学院硕士研究生.主要研究方向为遥感图像处理.E-mail: lhy_holly@njust.edu.cn" ]
  [ "韩海波男，1997年11月出生于浙江省杭州市.现为南京理工大学硕士研究生.主要研究方向为图像处理.E-mail: 1165530681@qq.com" ]
  [ "Jenny Qian Du 女，1971年出生于上海市.美国密西西比州立大学电气与计算机工程系教授.研究方向为高光谱遥感图像分析和应用、深度学习.E-mail: du@ece.msstate.edu" ]
  [ "吴泽彬（通讯作者）男，1981年出生于浙江省杭州市.南京理工大学计算机科学与工程学院教授.研究方向为图像处理、高性能计算.中国电子学会会员编号：E190032420M." ]
  [ "韦志辉男，1963年出生于江苏省淮安市.南京理工大学计算机科学与工程学院教授.研究方向为图像处理、数据科学方法与理论、计算机视觉.E-mail: gswei@njust.edu.cn" ]
- 基金信息：
  
  国家自然科学基金面上项目(61971223;61871226);中央高校基本科研业务费专项资金资助(JSGP202204)
- DOI：10.12263/DZXB.20210857
  中图分类号： TP391;
- 收稿：2021-07-06，
  
  修回：2022-09-05，
  
  纸质出版：2023-06-25
- 稿件说明：
移动端阅览
刘红毅,赵肖飞,韩海波等.基于光谱映射和细节注入的Pansharpening[J].电子学报,2023,51(06):1527-1540.

LIU Hong-yi,ZHAO Xiao-fei,HAN Hai-bo,et al.Pansharpening by Jointing Spectral Mapping and Detail Injection[J].ACTA ELECTRONICA SINICA,2023,51(06):1527-1540.
刘红毅,赵肖飞,韩海波等.基于光谱映射和细节注入的Pansharpening[J].电子学报,2023,51(06):1527-1540. DOI： 10.12263/DZXB.20210857.

LIU Hong-yi,ZHAO Xiao-fei,HAN Hai-bo,et al.Pansharpening by Jointing Spectral Mapping and Detail Injection[J].ACTA ELECTRONICA SINICA,2023,51(06):1527-1540. DOI： 10.12263/DZXB.20210857.

摘要

Pansharpening通过在多光谱图像中注入全色图像的空间细节，从而获得高分辨率的多光谱图像.但细节注入的同时，可能会引起光谱失真.为了提高融合图像的光谱保真，本文提出了一种空谱结合的Pansharpening方法.充分利用全色和多光谱图像之间潜在的光谱关系，对全色图像的光谱信息进行增强，继而将增强后的全色图像细节注入多光谱图像中，并建立了统一的变分Pansharpening模型，同时实现了融合图像的光谱信息保持和空间结构细节增强.在不同数据集上进行的数值实验表明，相比现有的Pansharpening方法，本文所提方法具有比较好的融合效果，尤其在光谱保真方面，更具有一定的优势.

Abstract

Pansharpening aims at improving spatial resolution and retaining spectral resolution for multispectral (MS) image with an aided panchromatic (PAN) image. Most of the existing methods improve the spatial resolution of MS image by injecting spatial details acquired from the PAN image

which may lead to spectral distortion. This paper proposes a joint spectral-spatial pansharpening method. By exploiting the underlying spectral correlation between the PAN and MS images

a spectral-enhanced PAN (SPAN) image is produced. Then the detail of the SPAN image is injected into the MS image

which leads to a unified variational pansharpening model to obtain a fused image with both spatial structure-enhanced and spectral-preserved. Experiments on different datasets confirm the effectiveness of the proposed pansharpening method

especially in the fidelity of spectra.

关键词

Keywords

references

RICHARDS J A , JIA X P . Using suitable neighbors to augment the training set in hyperspectral maximum likelihood classification [J]. IEEE Geoscience and Remote Sensing Letters , 2008 , 5 ( 4 ): 774 - 777 .

SUN L , MA C Y , CHEN Y J , et al . Adjacent superpixel-based multiscale spatial-spectral kernel for hyperspectral classification [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2019 , 12 ( 6 ): 1905 - 1919 .

SUN L , ZHAN T M , WU Z B , et al . Hyperspectral mixed denoising via spectral difference-induced total variation and low-rank approximation [J]. Remote Sensing , 2018 , 10 ( 12 ): 1956 .

HONG D F , YOKOYA N , GE N , et al . Learnable manifold alignment (LeMA): A semi-supervised cross-modality learning framework for land cover and land use classification [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2019 , 147 : 193 - 205 .

CHAVEZ JR P S , SIDES S C , ANDERSON J A . Comparison of three different methods to merge multiresolution and multispectral data: LANDSAT TM and SPOT panchromatic [J]. Photogrammetric Engineering & Remote Sensin , 1991 , 57 ( 3 ): 265 - 303 .

KANATSOULIS C I , FU X , SIDIROPOULOS N D , et al . Hyperspectral super-resolution: A coupled tensor factorization approach [J]. IEEE Transactions on Signal Processing , 2018 , 66 ( 24 ): 6503 - 6517 .

GHASSEMIAN H . A review of remote sensing image fusion methods [J]. Information Fusion , 2016 , 32 : 75 - 89 .

ALPARONE L , WALD L , CHANUSSOT J , et al . Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest [J]. IEEE Transactions on Geoscience and Remote Sensing , 2007 , 45 ( 10 ): 3012 - 3021 .

LONCAN L , DE ALMEIDA L B , BIOUCAS-DIAS J M , et al . Hyperspectral pansharpening: A review [J]. IEEE Geoscience and Remote Sensing Magazine , 2015 , 3 ( 3 ): 27 - 46 .

VIVONE G , ALPARONE L , CHANUSSOT J , et al . A critical comparison of pansharpening algorithms [C]// 2014 IEEE Geoscience and Remote Sensing Symposium . Quebec City : IEEE , 2014 : 191 - 194 .

MENG X C , SHEN H F , LI H F , et al . Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges [J]. Information Fusion , 2019 , 46 : 102 - 113 .

GARZELLI A . A review of image fusion algorithms based on the super-resolution paradigm [J]. Remote Sensing , 2016 , 8 ( 10 ): 797 .

XIONG Z W , SHI Z , LI H Q , et al . HSCNN: CNN-based hyperspectral image recovery from spectrally undersampled projections [C]// 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) . Venice : IEEE , 2018 : 518 - 525 .

HUANG W , XIAO L , WEI Z H , et al . A new pan-sharpening method with deep neural networks [J]. IEEE Geoscience and Remote Sensing Letters , 2015 , 12 ( 5 ): 1037 - 1041 .

MASI G , COZZOLINO D , VERDOLIVA L , et al . Pansharpening by convolutional neural networks [J]. Remote Sensing , 2016 , 8 ( 7 ): 594 - 615 .

WEI Y C , YUAN Q Q , MENG X C , et al . Multi-scale-and-depth convolutional neural network for remote sensed imagery pan-sharpening [C]// 2017 IEEE International Geoscience and Remote Sensing Symposium(IGARSS) . Fort Worth : IEEE , 2017 : 3413 - 3416 .

HE L , RAO Y Z , LI J , et al . Pansharpening via detail injection based convolutional neural networks [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2019 , 12 ( 4 ): 1188 - 1204 .

CHAVEZ P , KWATENG A . Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis [J]. Photogrammetric Engineering and Remote Sensing , 1989 , 55 ( 3 ): 339 - 348 .

LABEN C A , BROWER B V . Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening : 09/069232 [P]. 2000-01-04 .

AIAZZI B , BARONTI S , SELVA M . Improving component substitution pansharpening through multivariate regression of MS+Pan data [J]. IEEE Transactions on Geoscience and Remote Sensing , 2007 , 45 ( 10 ): 3230 - 3239 .

VIVONE G . Robust band-dependent spatial-detail approaches for panchromatic sharpening [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 9 ): 6421 - 6433 .

GARZELLI A , NENCINI F , CAPOBIANCO L . Optimal MMSE pan sharpening of very high resolution multispectral images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2008 , 46 ( 1 ): 228 - 236 .

CARPER W , LILLESAND T , KIEFFER R . The use of intensity-hue-saturation transformations for merging SPOT panchromaticand multispectral image data [J]. Photogrammetric Engineering and Remote Sensing , 1990 , 56 ( 4 ): 459 - 467 .

GHAHREMANI M , GHASSEMIAN H . Nonlinear IHS: A promising method for pan-sharpening [J]. IEEE Geoscience and Remote Sensing Letters , 2016 , 13 ( 11 ): 1606 - 1610 .

LI Q L , YANG X M , WU W , et al . Pansharpening multispectral remote-sensing images with guided filter for monitoring impact of human behavior on environment [J]. Concurrency and Computation: Practice and Experience , 2021 , 33 ( 4 ): e5074 .

JIAN L H , YANG X M , WU W , et al . Pansharpening using a guided image filter based on dual-scale detail extraction [J]. Journal of Ambient Intelligence and Humanized Computing , 2018 , 9 : 1 - 15 .

HE K M , SUN J , TANG X O . Guided image filtering [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 6 ): 1397 - 1409 .

KANG X D , LI S T , BENEDIKTSSON J A . Spectral-spatial hyperspectral image classification with edge-preserving filtering [J]. IEEE Transactions on Geoscience and Remote Sensing , 2014 , 52 ( 5 ): 2666 - 2677 .

LIU J G . Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details [J]. International Journal of Remote Sensing , 2000 , 21 ( 18 ): 3461 - 3472 .

MALLAT S G . A theory for multiresolution signal decomposition: The wavelet representation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 1989 , 11 ( 7 ): 674 - 693 .

STARCK J L , FADILI J , MURTAGH F . The undecimated wavelet decomposition and its reconstruction [J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society , 2007 , 16 ( 2 ): 297 - 309 .

SHENSA M J . The discrete wavelet transform: Wedding the a trous and Mallat algorithms [J]. IEEE Transactions on Signal Processing , 1992 , 40 ( 10 ): 2464 - 2482 .

CHOI M , KIM R Y , NAM M R , et al . Fusion of multispectral and panchromatic Satellite images using the curvelet transform [J]. IEEE Geoscience and Remote Sensing Letters , 2005 , 2 ( 2 ): 136 - 140 .

DO M N , VETTERLI M . The contourlet transform: An efficient directional multiresolution image representation [J]. IEEE Transactions on Image Processing , 2005 , 14 ( 12 ): 2091 - 2106 .

LAPORTERIE F , AMRAM O , FLOUZAT G , et al . Data fusion thanks to an improved morphological pyramid approach: Comparison loop on simulated images and application to SPOT 4 data [C]// IEEE 2000 International Geoscience and Remote Sensing Symposium . Honolulu : IEEE , 2002 : 2117 - 2119 .

AIAZZI B , ALPARONE L , BARONTI S , et al . MTF-tailored multiscale fusion of high-resolution MS and pan imagery [J]. Photogrammetric Engineering & Remote Sensing , 2006 , 72 ( 5 ): 591 - 596 .

VIVONE G , RESTAINO R , CHANUSSOT J . Full scale regression-based injection coefficients for panchromatic sharpening [J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society , 2018 , 27 ( 7 ): 3418 - 3431 .

VIVONE G , MARANO S , CHANUSSOT J . Pansharpening: Context-based generalized Laplacian Pyramids by robust regression [J]. IEEE Transactions on Geoscience and Remote Sensing , 2020 , 58 ( 9 ): 6152 - 6167 .

LIU P F , XIAO L , LI T . A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 3 ): 1788 - 1802 .

BALLESTER C , CASELLES V , IGUAL L , et al . A variational model for P+XS image fusion [J]. International Journal of Computer Vision , 2006 , 69 ( 1 ): 43 - 58 .

JIANG C , ZHANG H Y , SHEN H F , et al . A practical compressed sensing-based pan-sharpening method [J]. IEEE Geoscience and Remote Sensing Letters , 2012 , 9 ( 4 ): 629 - 633 .

GUO M , ZHANG H Y , LI J Y , et al . An online coupled dictionary learning approach for remote sensing image fusion [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2014 , 7 ( 4 ): 1284 - 1294 .

FU X Y , LIN Z H , HUANG Y , et al . A variational pan-sharpening with local gradient constraints [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach : IEEE , 2020 : 10257 - 10266 .

ZHU X X , BAMLER R . A sparse image fusion algorithm with application to pan-sharpening [J]. IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 5 ): 2827 - 2836 .

AHARON M , ELAD M , BRUCKSTEIN A . K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation [J]. IEEE Transactions on Signal Processing , 2006 , 54 ( 11 ): 4311 - 4322 .

HUANG W , XIAO L , LIU H Y , et al . Hyperspectral imagery super-resolution by compressive sensing inspired dictionary learning and spatial-spectral regularization [J]. Sensors , 2015 , 15 ( 1 ): 2041 - 2058 .

LI S T , YIN H T , FANG L Y . Remote sensing image fusion via sparse representations over learned dictionaries [J]. IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 9 ): 4779 - 42016 .

HE W , ZHANG H Y , ZHANG L P , et al . Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration [J]. IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 1 ): 178 - 188 .

XU Y , WU Z B , CHANUSSOT J , et al . Nonlocal patch tensor sparse representation for hyperspectral image super-resolution [J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society , 2019 , 28 ( 6 ): 3034 - 3047 .

DURAN J , BUADES A , COLL B , et al . Pansharpening by a nonlocal channel-decoupled variational method [C]// 2016 IEEE International Conference on Image Processing(ICIP) . Phoenix : IEEE , 2016 : 4339 - 4343 .

ZHANG H Y , HE W , ZHANG L P , et al . Hyperspectral image restoration using low-rank matrix recovery [J]. IEEE Transactions on Geoscience and Remote Sensing , 2014 , 52 ( 8 ): 4729 - 4743 .

LIU H Y , SUN P P , DU Q , et al . Hyperspectral image restoration based on low-rank recovery with a local neighborhood weighted spectral-spatial total variation model [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 3 ): 1409 - 1422 .

ZHOU J , KWAN C , BUDAVARI B . Hyperspectral image super-resolution: A hybrid color mapping approach [J]. Journal of Applied Remote Sensing , 2016 , 10 ( 3 ): 035024 .

VITALE S , SCARPA G . A detail-preserving cross-scale learning strategy for CNN-based pansharpening [J]. Remote Sensing , 2020 , 12 ( 3 ): 348 .

WANG D , LI Y , MA L , et al . Going deeper with densely connected convolutional neural networks for multispectral pansharpening [J]. Remote Sensing , 2019 , 11 ( 22 ): 2608 .

ZHAO X F , LIU H Y , ZHANG J , et al . A spectral mapping based intensity modulation for pan-sharpening [C]// IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium . Yokohama : IEEE , 2019 : 843 - 846 .

LI D , SHEN J , LU Y , et al . A class of alternating linearization algorithms for nonsmooth convex optimization [J]. Acta Mathematicae Applicatae Sinica , English Series, 2019 , 35 ( 2 ): 435 - 443 .

GAO X , CAI X J , HAN D R . A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems [J]. Journal of Global Optimization , 2020 , 76 ( 4 ): 863 - 887 .

BOLTE J , SABACH S , TEBOULLE M . Proximal alternating linearized minimization for nonconvex and nonsmooth problems [J]. Mathematical Programming , 2014 , 146 ( 1 ): 459 - 494 .

WALD L , RANCHIN T , MANGOLINI M . Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images [J]. Photogrammetric Engineering and Remote Sensing , 1997 , 63 ( 6 ): 691 - 699 .

OTAZU X , GONZALEZ-AUDICANA M , FORS O , et al . Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods [J]. IEEE Transactions on Geoscience and Remote Sensing , 2005 , 43 ( 10 ): 2376 - 2385 .

YUHAS R H , GOETZ ALEXANDER F H , BOARDMAN J W . Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm [C]// Summaries of the Third Annual JPL Airborne Geoscience Workshop . California : Jet Propulsion Laboratory , 1992 : 147 - 149 .

WALD L . Data fusion: definitions and architectures - fusion of images of different spatial resolutions [EB/OL]. ( 2002 )[2021]. https://hal-mines-paristech.archives-ouvertes.fr/hal-00464703 https://hal-mines-paristech.archives-ouvertes.fr/hal-00464703 .

WANG Z , BOVIK A C . A universal image quality index [J]. IEEE Signal Processing Letters , 2002 , 9 ( 3 ): 81 - 84 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于非下采样三通道不可分对称小波的多光谱图像融合

基于采样二通道不可分小波的多光谱图像融合

基于变分的多尺度遥感图像融合算法