基于光谱映射和细节注入的Pansharpening

doi:10.12263/DZXB.20210857

电子学报

• •

基于光谱映射和细节注入的Pansharpening

刘红毅¹, 赵肖飞¹, 韩海波¹, Du Jenny Qian², 吴泽彬³, 韦志辉³

^1.南京理工大学数学与统计学院，江苏南京 210094
^2.密西西比州立大学电气与计算机工程系，密西西比州斯塔克维尔 39762
^3.南京理工大学计算机工程与科学学院，江苏南京 210094

收稿日期:2021-07-06 修回日期:2022-09-05 出版日期:2023-03-06
- 通讯作者:
- 吴泽彬
- 作者简介:
- 刘红毅女，1974年出生于陕西省西安市.南京理工大学数学与统计学院副教授.研究方向为基于图像处理与分析、计算机视觉.E-mail: hyliu@njust.edu.cn
  赵肖飞女，1995年出生于河北省沧州市.南京理工大学数学与统计学院硕士研究生.主要研究方向为遥感图像处理.E-mail: lhy_holly@njust.edu.cn
  韩海波男，1997年11月出生于浙江省杭州市.现为南京理工大学硕士研究生.主要研究方向为图像处理.E-mail: 1165530681@qq.com
  Jenny Qian Du 女，1971年出生于上海市.美国密西西比州立大学电气与计算机工程系的教授.研究方向为高光谱遥感图像分析和应用、深度学习.E-mail: du@ece.msstate.edu
  吴泽彬（通讯作者）男，1981年出生于浙江省杭州市.南京理工大学计算机科学与工程学院的教授.研究方向为图像处理、高性能计算.
  韦志辉男，1963年出生于江苏省淮安市.南京理工大学计算机科学与工程学院的教授.研究方向为图像处理、数据科学方法与理论、计算机视觉.E-mail: gswei@njust.edu.cn
- 基金资助:
- 国家自然科学基金面上项目(61971223);中央高校基本科研业务费专项资金资助(JSGP202204)

Pansharpening by Jointing Spectral Mapping and Detail Injection

LIU Hong-yi¹, ZHAO Xiao-fei¹, HAN Hai-bo¹, DU Jenny Qian², WU Ze-bin³, WEI Zhi-hui³

^1.School of Mathematics and Statistics，Nanjing University of Science and Technology，Nanjing，Jiangsu 210094，China
^2.Department of Electrical and Computer Engineering，Mississippi State University，Mississippi State，MS 39762，America
^3.School of Computer Science and Engineering，Nanjing University of Science and Technology，Nanjing，Jiangsu 210094，China

Received:2021-07-06 Revised:2022-09-05 Online:2023-03-06
- Corresponding author:
- WU Ze-bin
- Supported by:
- General program of National Natural Science Foundation of China(61971223);Special Funds for Basic Scientific Research Business Expenses of Central Universities(JSGP202204)

摘要/Abstract

摘要：

Pansharpening通过在多光谱图像中注入全色图像的空间细节，从而获得高分辨率的多光谱图像.但细节注入的同时，可能会引起光谱失真.为了提高融合图像的光谱保真，本文提出了一种空谱结合的Pansharpening方法.充分利用全色和多光谱图像之间潜在的光谱关系，对全色图像的光谱信息进行增强，继而将增强后的全色图像细节注入到多光谱图像中，并建立了统一的变分Pansharpening模型，同时实现了融合图像的光谱信息保持和空间结构细节增强.在不同数据集上进行的数值实验表明，相比现有的Pansharpening方法，本文所提出的方法具有比较好的融合效果，尤其在光谱保真方面，更具有一定的优势.

关键词: Pansharpening, 多光谱图像, 全色图像, 光谱映射, 细节注入

Abstract:

Pansharpening aims at improving spatial resolution and retaining spectral resolution for multispectral(MS) image with an aided panchromatic(PAN) image. Most of the existing methods improve the spatial resolution of MS image by injecting spatial details acquired from the PAN image, which may lead to spectral distortion. This paper proposes a joint spectral-spatial pansharpening method. By exploiting the underlying spectral correlation between the PAN and MS images, a spectral-enhanced PAN(SPAN) image is produced. Then the detail of the SPAN image is injected into the MS image, which leads to a unified variational pansharpening model to obtain a fused image with both spatial structure-enhanced and spectral-preserved. Experiments on different datasets confirm the effectiveness of the proposed pansharpening method, especially in the fidelity of spectra.

Key words: Pansharpening, multispectral image, panchromatic image, spectral mapping, detail injection

中图分类号:

TP391

刘红毅, 赵肖飞, 韩海波, 等. 基于光谱映射和细节注入的Pansharpening[J]. 电子学报, DOI: 10.12263/DZXB.20210857.

Hong-yi LIU, Xiao-fei ZHAO, Hai-bo HAN, et al. Pansharpening by Jointing Spectral Mapping and Detail Injection[J]. Acta Electronica Sinica, DOI: 10.12263/DZXB.20210857.

参考文献 24

1	RICHARDS J A, JIA X P. Using suitable neighbors to augment the training set in hyperspectral maximum likelihood classification[J]. IEEE Geoscience and Remote Sensing Letters, 2008, 5(4): 774-777.
2	SUN L, MA C Y, CHEN Y J, et al. Adjacent superpixel-based multiscale spatial-spectral kernel for hyperspectral classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(6): 1905-1919.
3	SUN L, ZHAN T M, WU Z B, et al. Hyperspectral mixed denoising via spectral difference-induced total variation and low-rank approximation[J]. Remote Sensing, 2018, 10(12): 1956.
4	HONG D F, YOKOYA N, GE N, et al. Learnable manifold alignment (LeMA): A semi-supervised cross-modality learning framework for land cover and land use classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 147: 193-205.
5	CHAVEZ JR P S, SIDES S C, ANDERSON J A. Comparison of three different methods to merge multiresolution and multispectral data: LANDSAT TM and SPOT panchromatic[J]. Photogrammetric Engineering & Remote Sensin, 1991, 57(3): 265-303.
6	KANATSOULIS C I, FU X, SIDIROPOULOS N D, et al. Hyperspectral super-resolution: A coupled tensor factorization approach[J]. IEEE Transactions on Signal Processing, 2018, 66(24): 6503-6517.
7	GHASSEMIAN H. A review of remote sensing image fusion methods[J]. Information Fusion, 2016, 32: 75-89.
8	ALPARONE L, WALD L, CHANUSSOT J, et al. Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest[J]. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10): 3012-3021.
9	LONCAN L, DE ALMEIDA L B, BIOUCAS-DIAS J M, et al. Hyperspectral pansharpening: A review[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3): 27-46.
10	VIVONE G, ALPARONE L, CHANUSSOT J, et al. A critical comparison of pansharpening algorithms[C]//2014 IEEE Geoscience and Remote Sensing Symposium. Quebec City: IEEE, 2014: 191-194.
11	MENG X C, SHEN H F, LI H F, et al. Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges[J]. Information Fusion, 2019, 46: 102-113.
12	GARZELLI A. A review of image fusion algorithms based on the super-resolution paradigm[J]. Remote Sensing, 2016, 8(10): 797.
13	XIONG Z W, SHI Z, LI H Q, et al. HSCNN: CNN-based hyperspectral image recovery from spectrally undersampled projections[C]//2017 IEEE International Conference on Computer Vision Workshops(ICCVW). Venice: IEEE, 2018: 518-525.
14	HUANG W, XIAO L, WEI Z H, et al. A new pan-sharpening method with deep neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5): 1037-1041.
15	MASI G, COZZOLINO D, VERDOLIVA L, et al. Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8(7): 594-615.
16	WEI Y C, YUAN Q Q, MENG X C, et al. Multi-scale-and-depth convolutional neural network for remote sensed imagery pan-sharpening[C]//2017 IEEE International Geoscience and Remote Sensing Symposium(IGARSS). Fort Worth: IEEE, 2017: 3413-3416.
17	HE L, RAO Y Z, LI J, et al. Pansharpening via detail injection based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(4): 1188-1204.
18	CHAVEZ P, KWATENG A. Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis[J]. Photogrammetric Engineering and Remote Sensing, 1989, 55(3): 339-348.
19	LABEN C A, BROWER B V. Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening: 09/069232[P]. 2000-01-04.
20	AIAZZI B, BARONTI S, SELVA M. Improving component substitution pansharpening through multivariate regression of MS+Pan data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10): 3230-3239.
21	VIVONE G. Robust band-dependent spatial-detail approaches for panchromatic sharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6421-6433.
22	GARZELLI A, NENCINI F, CAPOBIANCO L. Optimal MMSE pan sharpening of very high resolution multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(1): 228-236.
23	CARPER W, LILLESAND T, KIEFFER R. The use of intensity-hue-saturation transformations for merging SPOT panchromaticand multispectral image data[J]. Photogrammetric Engineering and Remote Sensing,1990, 56(4), 459-467.
24	GHAHREMANI M, GHASSEMIAN H. Nonlinear IHS: A promising method for pan-sharpening[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606-1610.

[1]	秦福强, 王丽芳. 基于变分的多尺度遥感图像融合算法[J]. 电子学报, 2020, 48(6): 1084-1090.
[2]	张永梅, 季艳, 马礼, 张睿, 李洁琼, 熊焰. 遥感图像建筑物识别及变化检测方法[J]. 电子学报, 2014, 42(4): 653-657.
[3]	刘斌, 祝青, 胡福强, 刘维杰. 基于采样二通道不可分小波的多光谱图像融合[J]. 电子学报, 2013, 41(4): 710-716.
[4]	刘斌;彭嘉雄. 基于非下采样三通道不可分对称小波的多光谱图像融合[J]. 电子学报, 2011, 39(5): 1094-1099.
[5]	陈瑞;孙静宇;林喜荣;丁天怀. 利用多光谱图像的伪造虹膜检测算法[J]. 电子学报, 2011, 39(3): 710-713.
[6]	佘二永;王润生;徐学文. 基于统计模型的遥感图像多分辨融合方法[J]. 电子学报, 2008, 36(3): 490-493.

基于光谱映射和细节注入的Pansharpening

Pansharpening by Jointing Spectral Mapping and Detail Injection

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献 24

相关文章 6

编辑推荐

Metrics